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ABSTRACT 


The purpose of this thesis is to estimate the potential performance improvement in 
sustaining engineering (SE) when an Open Architecture (OA) approach to system 
development is used. Its basis is that in Integrated Warfare Systems (IWS) acquisition, 
eighty percent of total life-cycle costs occur during the operation and support phase. This 
statistic demonstrates the necessity of measuring how the OA approach will affect 
software upgrade and maintenance process for the AEGIS IWS Eife Cycle. Using the OA 
approach, advances in distance support and monitoring, and maintenance free operating 
periods are possible, and this is significant in supporting the need to reduce costs and 
manpower while improving performance. To estimate the potential (Return on 
Investment) ROI that an OA approach might enable for SE in the form of software 
maintenance and upgrade, this thesis will apply the Knowledge Value Added (KVA) 
methodology to establish the baseline, “As Is,” configuration of the current solutions in 
AEGIS. The KVA analysis will yield the ROTs and the current models for the approach 
to software maintenance and upgrade. Based on the assumptions of OA design for 
original system development, new approaches to distance and maintenance and 
monitoring will be explored in “To Be” solutions, and the ROIs will be estimated. The 
“To Be” solutions are rooted in the assumptions of MEOP and ARCI, and the results 
indicate that these solutions yield a potential improvement of 720% and a cost saving of 
$365,104.63 over the current methodology for just one ship. Eor all ships using AEGIS, 
ROI improves by 71,967% with a cost savings of $26,543,824.56. The conclusion is that 
OA enables extension of these best practice approaches to AEGIS maintenance and 
upgrade solutions. 
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I. 


INTRODUCTION 


A. PURPOSE 

The need has always existed aboard ship to maintain operations of specific 
organic assets while at sea, and as the United States Navy rapidly advances towards 
reduced manning, Integrated Warfare Systems (IWS) will require a new approach to 
Sustaining Engineering (SE) for software maintenance and upgrade solutions if a smaller 
crew is to perform at the same caliber. The value of sustaining engineering and creating 
more efficient software upgrades can be realized by increased time spent on mission 
efforts and decreased time spent on solving IWS anomalies. The more efficient system 
design of the future can adapt to the requirements through open architecture (OA), and 
when designers use an OA approach it enables innovation without major efforts on the 
part of the ship’s crew. 

A previous study conducted at Naval Postgraduate School by Capt. Joseph 
Uchytil entitled, “Assessing the Operational Value of Situational Awareness of AEGIS 
and Ship Self-Defense System (SSDS) Platforms through the Application of the 
Knowledge Value Added (KVA) Methodology,” demonstrated that KVA could be used 
to estimate the performance of an OA implementation in terms of a Return on Investment 
(ROI). While Capt. Uchytil’s research focused on benefits derived from the warfighter’s 
perspective, the purpose of this research is to generate and assess the impact of an OA 
development and acquisition approach to SE in IWS. 

By extending the focus of this OA study from the warfighter to the acquirer and 
system developer the analysis provides a more complete view of the whole system 
development lifecycle and the potential of OA to improve the performance of the cycle in 
the SE processes. Due to the scope of this study it is more likely for the SE portion of the 
life cycle to achieve the highest potential productivity which, in turn, helps to make sure 
they exploit the benefits of OA in this part of the life cycle which is developer and 
acquirer intensive. 
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B. BACKGROUND 


The benefit of open architecture from the developer and acquirer point-of-view is 
that it creates greater flexibility by introducing additional technologies and capabilities to 
the fleet which closed systems of the past have failed to introduce after procurement. This 
is because closed systems are, typically, not as amenable to rapid upgrades as open 
systems. Current systems need to be fluid and dynamic to respond to and anticipate the 
anomalies encountered on ships. Ergo, it is no longer practical for software maintenance 
support and upgrades to operate within closed systems because they are difficult and slow 
to upgrade, have limited interoperability, and upgrades must be done with the same 
vendor. Additionally, it is hard to maintain proprietary systems because of their 
interdependencies in code and software. Due to lifecycle time and cost constraints, OA, 
which offers faster business and system models to the acquirer and developer and 
independent coding, should be effectively used in order to promote the future view of a 
Navy which will operate within the Global Information Grid (GIG). The Global 
Information Grid (GIG) will create an informative and integrated environment to pass the 
information.! This will require integration of several parts, and it will require those parts 
to be fully operational, usable, and easily upgraded under reduced manning and joint 
operational conditions. One of the enablers of GIG will be the effectiveness of open 
architecture which will allow for faster integration when applied to sustaining 
engineering i.e. the software maintenance support and upgrade process. 

C. RESEARCH OBJECTIVES 

The objective of this research is to analyze the potential benefits of open 
architecture from the developers and acquirers perspective in the SE process for the 
AEGIS weapons system. This will be achieved through the KVA approach that will 
provide the analytical framework to assess the added value of the open architecture 
approach to software maintenance support and upgrade solutions to the developer and 
acquirer. 


1 Clark, V. 2002. 
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The KVA approach provides the static ROI analysis models which serve as the 
input for Real Options which will be conducted in further research. By placing both 
related and unrelated elements in a single unit, the value of both objects can be compared 
in one lump figure. Please refer to the appendix for a more detailed description of the 
KVA methodology. 

D. RESEARCH QUESTIONS 

Since the measure of effectiveness of a ship in the fleet does not rely solely on 
monetary cost savings, but rather on the productivity and mission accomplishment, the 
knowledge value-added approach can be applied to produce a return on investment (ROI) 
that will serve as a measure of productivity. Possible models that would increase the 
productivity of software maintenance support and upgrade solutions in the AEGIS 
weapons system can be explored after a baseline of the current system is established. 

This thesis will provide “To-Be” scenarios for using an open architecture 
approach to meet the demands of the future Navy with regards to sustaining engineering. 
The secondary research will explore the Department of Defense and Department of the 
Navy initiatives for Open Architecture, Open Architecture Computing Environments, 
Services Oriented Architecture Solutions, Distance Support Solutions, and current “best 
practices” examples in software upgrades in military environments. 

Our analysis will answer the following research questions: 

* Using IBM’s Component Business Modeling (CBM), what are the areas 
of highest concern and cost in the AEGIS weapons system as they relate to 
sustaining engineering? 

* What are the “best practice” examples of sustaining engineering in the 
commercial or military environment and how do they improve the 
processes of system development and acquisition life-cycles? What are 
the best examples of SOA and distance support systems? 
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* What is the benefit of OA in the process of sustaining engineering? We 
will apply the KVA methodology to the areas of highest concern as 
identified by IBM in their CBM to address these questions. . 

* What overall numerical effect does OA have on Sustaining Engineering 
and is it appreciable and low-risk enough to system development and the 
DoD acquisition lifecycle? 

This research will provide decision makers with a structured analysis of employing open 
architecture to improve productivity within sustaining engineering and software upgrades 
for Integrated Warfare Systems (IWS). 

E. METHODOLOGY 

We will employ the case example method when conducting our research. We will 
focus on exploring various avenues of improvement associated with sustaining 
engineering, specifically the software maintenance and upgrade processes for the AEGIS 
system. A KVA analysis will be conducted on the system in the “As-Is” configuration. 
This will serve as a static baseline analysis and will be used to generate the To-Be” 
models. The KVA and follow-on RO analysis in future research will then be conducted 
for the software maintenance and upgrade process that employs an OA framework. Both 
analyses will be conducted with the help of AEGIS subject matter experts. The KVA 
analysis associated with the “As-Is” and the “To-Be” (employing an OA framework) 
systems will produce an ROI for each process model. The ROI associated with each 
process model will be compared in further research to determine the impact of OA as a 
viable solution to improving sustaining engineering and the software maintenance and 
upgrade processes associated with AEGIS. The follow on RO analysis will identify 
options associated with each process model including valuation options, cost options and 
risk options. The results will be compared and evaluated and the benefit of OA to 
sustaining engineering and the software upgrade processes for Integrated Weapons 
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Systems (IWS) will be determined. The researeh will offer reeommendations on how to 
improve sustaining engineering and the software upgrade proeesses in the eontext of OA 
systems. 

F. SCOPE 

The seope of this thesis will be to specifically prescribe an operational value to 
the improvement of the software maintenance and upgrade procedures of AEGIS using 
OA. This research highlights the inherent value of knowledge capital in a system by 
using KVA methodologies and it emphasizes the need to introduce OA solutions to many 
more sustaining engineering processes aboard ships which will undergo reduced manning 
and be expected to achieve “decision superiority’ in the future. 

G. THESIS ORGANIZATION 

The chapter organization will be: Chapter I will give a general overview of the 
purpose and intended methods and scope of the thesis. Chapter II will provide secondary 
research and background information on open architecture aims, OACE, SOA, distance 
support solutions, and best practice examples. Chapter III will consist of the KVA 
analysis giving the resulting figures and charts. Chapter IV will discuss the results from 
the KVA analysis and the implications of the current “As-Is” state of AEGIS 
maintenance and then explore the “To-Be” results. Chapter V will present conclusions 
from the research that was conducted. Chapter VI will recommend further research that 
can be conducted to continue the process of refining sustaining engineering and software 
upgrades within the context of open architecture in the fleet. 
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II. LITERATURE REVIEW: SOETWARE UPGRADES AND 
MAINTENANCE SOLUTIONS IN THE OPEN ARCHITECTURE 

ENVIRONMENT 


A. CBM THEORY AND SUSTAINING ENGINEERING 

The driving factor that has promoted the need for improvement of Sustaining 
Engineering (SE) in software maintenance and upgrade is the Component Business 
Model (CBM) process conducted by IBM in June of 2006. Component Business 
Modeling is an “IBM-developed technique for representing an enterprise as non¬ 
overlapping business components in order to identify opportunities for innovation and/or 
improvement.”^ In a CBM, a business component is a group of resources, people, and 
technology that have the necessary information to deliver value from functional 
performance.3 The final result of the grouping of business components is visually 
represented in a component business map which hones in on the essential foundational 
blocks of the organization. Table 1 is the final component business map for the 
breakdown of AEGIS in Program Executive Office Integrated Warfare Systems (PEO 
IWS) for Eiscal Year 2007. 


2 Pavlick, Tim 2005; 7. 

3 Ibid., 7. 
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Table 1. CBM Map Identifying Sustaining Engineering as “Hot Component”^ 
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“Hot components,” or components that are worth further examination, are 
represented by a star. They are identified as hot components based on attributes selected 
as important to the organization being assisted through the analysis. In the case of this 
CBM effort, there were three criteria selected: investment of total budget (green), number 
of efforts required for the task (yellow), and color of money (orange).^ For the sustaining 
engineering category, it has a medium percentage of the PEO IWS budget, a high number 
of efforts (greater than six), and two colors of money involved. The colors of money, or 
the money which is procured and used for specific acquisitions, are in the areas of 
Operation and Maintenance (OMN) and Ship Building and Conversion (SCN). The 
horizontal axis represents a key competency, or one which requires similar skills and 
capabilities while the vertical axis represents accountability levels. SE is “System 
Sustainment and Disposal” competency, in which the “executing,” branch, the branch 
that does the work, is accountable. 

In addition to SE being identified as a “hot component,” the Operations and 
Support (O&S) phase of a system’s life cycle is often represented to incur 80% of the 
total life cycle costs of a system. According to an article published in Program Managers 
Magazine, weapons system sustainment consumes about 80 percent of logistics 
resources, or approximately 64B per year.”6 With such a large factor of the total life 
cycle costs being focused in this life cycle phase, along with the CBM results, it is 
reasonable to examine Sustaining Engineering for ways to make it more efficient. IBM 
also anticipates that a large cost component within O&S is SE.^ In table 2 this is evident. 
In fact, this is the only starred area on the CBM diagram in which all colors of money 
will increase spending. KVA will seek to give decision makers a tool-set for making the 
vast amount of spending on SE more efficient through the use of OA. 


5 Shannon, James 2006; 11. 

6 Kratz, Louis A. 2002; 2. 

^ Shannon, James 2006; 16. 


9 




Table 2. Sustaining Engineering Color of Money Cost Increase ^ 


B. OPEN ARCHITECTURE ENVIRONMENT AND TENETS 

To achieve efficiency in software upgrade and maintenance it is necessary to 
eliminate current inadequacies in implementing open architecture. Department of 
Defense systems, according to a report release in 2006 by the Government Accountability 
Office, continue to lag behind in interoperability, even though the Program Executive 
Office, Integrated Warfare Systems (PEO IWS) was established in 2002 and was in 
charge of executing OA.9 Perhaps this is a result of the well known and frequently 
addressed security concerns involved in implementing OA in weapons systems, such as 
malicious code, computer viruses or system latency; however, the civilian sector grasped 
the concept quickly and they also have a need for security. Even banks, for the most part, 
operate with the framework of OA. Some banks even operate with Service Oriented 
Architecture which will be defined and discussed at the conclusion of the literature 
review. 

To implement OA, it is necessary to understand some basic concepts. In a general 
sense, OA is realized through rapid change and fluid upgrades and solutions. According 
to the Deputy Chief of Naval Operations, the requirements for OA implementation are as 
follows: modular design and design disclosure, reusable application software. 


^ Shannon, James 2006; 16. 

9 Government Accountability Office 2006. 
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interoperable joint warfighting applications and secure information exchange, life cycle 
affordability, encouraging competition and collaboration, scalability and portability 

1. Modular Design and Design Disclosure 

Modularity is the concept of decomposing a system into transparent 
subcomponents These subcomponents are operable without relying on another aspect 
of the system; hence, they can rapidly change and allow for interactions with numerous 
systems. The underlying goal of decomposition, in the case of modularity, is to allow for 
the independent upgrade of each of the smallest subcomponents while leaving the 
complete system operable. With modular design and design disclosure, multiple 
competitors can participate and innovation flourishes as each subcomponent is 
independently tried and tested. 

2. Reusable Application Software 

Reuse allows a system to use the same components and code that have been used 
across other platforms. ^2 In the case of application software, a database of segments of 
code that worked for the tracking device of one platform can be shared when creating 
other tracking devices. This would be a database that would be continually updated with 
components and segments of code that could have potential use in other areas. These 
components can be used interchangeably with other components without affecting the 
system in its entirety. This idea is revolutionary for coding and software upgrade in much 
the same way that “interchangeable parts” revolutionized the assembly lines of the 1920’s 
with increased output and increased revenue. Disclosure of the design of application 
software would also be necessary for evolutionary improvement in future upgrades. 


Chief of Naval Operations 2005. 

11 Coronado Mondragon, Christian E. 2006; 247. 

12 Chief of Naval Operations 2005. 

13 Ibid. 
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Interoperable Joint Warfighting Applications and Secure Information 
Exchange 


This particular tenet ensures that aeross a wide variety of systems the same 
information and applieations ean be shared. It involves commonality of services, 
warfighting applieations, and information assuranee and requires these eommonalities to 
be essential for the basie design elements of any new system, 

4. Life Cycle Affordability 

This tenet ineludes all phases of the life eyele from design and requirements 
gathering to delivery and testing. Since the primary eoneern of this thesis is the sustaining 
engineering portion, and sinee SE is sueh a large ehuek of the life eycle eosts, the results 
eould direetly benefit the implementation of OA with respeet to life cyele affordability. 

5. Encouraging Competition and Collaboration 

OA naturally encourages competition and collaboration because unlike closed 
systems many different systems can be integrated to complete upgrades or create a new 
system. That is not to say that proprietary systems did not contain many different parts 
that required different companies to collaborate but they were less likely to constantly 
create an environment of competition and innovation because some of the contracts were 
sole-source. Sole source contracts being those which restrict Full Open Competition as it 
is a non-competitive procurement process in which solicitation is only with one source,!^ 

6. Scalability 

Scalability encompasses the introduction of new functionalities into a system 
without procuring a whole new system to do the same job. An example of scalability is 
the method of increasing bandwidth during the holiday season to allow for faster 
transactions during a season of heightened traffic. 


Chief of Naval Operations 2005. 

1^ Department of Defense 5000.1 2003. 
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7. PortabUity 


Portability is the ability of the software or hardware and the users to easily 
integrate into different platforms. It requires source code to make transitions between 
hardware and software and requires the switch to be rapidly and smoothly accomplished. 

C. LEGACY SYSTEMS AND THEIR EFFECT ON SOFTWARE UPGRADES 

AND MAINTENANCE 

For the most part, closed systems of the past contain software which is designed 
for the purpose of supporting the computing hardware. When proprietary systems do 
need upgrades, computer code must change as well, but their unique design sometimes 
makes software upgrades financially and technically prohibitive. Programs such as these 
delay the time to introduce new technologies to the fleet and increase the total life-cycle 
cost of the system. Table 1, shown below, lists the contrasting characteristics between 
closed systems and open systems. Most important to this research are three points: 

* That expansion and upgrade of closed systems requires more time, effort, 
and money than the open systems 

* That closed systems are less adaptable to changes in threats and new 
technologies. 

* That closed systems are focused mostly on development cost and meeting 
the present mission while open systems focus on the total costs of 
ownership, sustainment, and growth of the system. 
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Closed System Characteristics 

Open System Characteristics 

Use of closely held, private interfaces, 
languages, data formats and protocols 
(government or vendor unique standards) 

Use of publicly available and widely used 
interfaces, languages, data formats and 
protocols 

Critical importance is given to unique 
design and implementation 

Critical importance is given to interfaces 
management, and widely used conventions 

Less emphasis on modularity 

Heavy emphasis on modularity 

Vendor and technology dependency 

Vendor and technology independence 

Minimization of the number of 
implementations 

Minimization of the number of types of 
interfaces 

Difficult and more costly integration 

High degree of portability, connectivity, 
interoperability, and scalability 

Use of sole-source vendor 

Use of multiple vendors 

Expansion and upgrading usually requires 
considerable time, money and effort 

Easier, quicker and less expensive 
expansion and upgrading 

Higher total ownership cost 

Lower total ownership cost 

Slower and more costly technology to 
transfer 

Technology transfer is faster and less 
costly 

Components, interfaces, standards, and 
implementations are selected sequentially 

Components, interfaces, standards, and 
implementations are selected interactively 

Systems with shorter life expectancy 

Systems with longer life expectancy 

Use of individual company preferences to 
set and maintain specifications 

Use of group consensus process to 
maintain interface specifications 

Less adaptable to change in threats and 
technologies 

More adaptable to evolving threats and 
technologies 

Focusing mostly on development cost and 
meeting present mission 

Focusing on total costs of ownership, 
sustainment and growth 

User as the producer of system 

User as the consumer of components 

Rigid and slow system of influence and 
control 

Real time and cybernetic system of 
influence and control 

Adversarial relationship with prime 
contractors/supplier/vendors 

Symbiotic relationship with prime 
contractors/suppliers/vendors 

Mostly confined to traditional suppliers 

Non-traditional suppliers can compete 

Simple conformance testing 

Very challenging conformance testing 


Table 3. Open Systems v. Closed Systems 


16 Azani, C. 2001; 1. 
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Legacy systems also have a specific computational power limitation. Systems like 
the AEGIS 6Ph3 radar processing has software, which relies on the military standard 
computer, UYK-43, which was sole-source contracted to Lockheed Martin in 1980. 
Such systems cannot keep up with the steadily increasing computational power in the 
commercial sector. The negative effect on current, closed systems is magnified because 
they are fast becoming obsolete while the benefits of OA are not realized. This research 
seeks to shift the focus of the AEGIS system software upgrades to increase the overall 
value of the system both monetarily, operationally, and from a user’s perspective by 
proving the worth of the implementation of open architecture in software maintenance 
and development where it is most amenable. 

D. OPEN ARCHITECTURE COMPUTING ENVIRONMENT 


As stated previously, the high costs of computer program maintenance and 
development are attributed to obsolescence, frequently needed changes, and proprietary 
systems which contain software applications that are closely linked to the backbone of 
system operation. Maintenance and development of such software could adversely affect 
the system as a whole, thus making it less amenable to any type of change which, 
hypothetically, is avoidable in an age with rapid development of commercial off-the-shelf 
(COTS) technology. 

According to the directive for Network Centric Warfare, the open architecture 
concept will be applied not only to hardware but also to software and the computing 
environment. Open Architecture Computing Environment (OACE) is, at its essence, the 
application of open architecture to computing systems so that over the life of the platform 
changes can be made with commercial technologies that will rapidly meet the changing 
demands of reduced manning source. Closed systems of the past reduced the ability of 
developers to modernize the system and to provide maintenance solutions for underlying 
problems. They also robbed the acquisition field of competitive contracts, as their field of 
suppliers was limited. In existing proprietary systems, obsolescence and the inability to 
introduce upgrades has decreased the overall value of acquiring the system. Vice 
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Admiral John Nathan said to the House Armed Services Committee in 2003, "By 
pursuing an Open Architecture and an Open Architecture Computing Environment based 
on mainstream COTS technologies, systems and standards, we can avoid the high cost of 
maintaining and upgrading multiple legacy computing systems that quickly become 
obsolete and are not responsive to changes in war-fighting requirements.Additionally, 
DoD directive 5000.1 states, “a modular, open-systems approach shall be employed 
where feasible," and a memorandum in April 2004 from the Under Secretary of Defense 
for Acquisition, Technology, and Logistics, expands on that language by requiring that 
all programs are subject to a milestone review brief their modular open-systems 

approach. 19 

1. OACE Shift 

The move to the OACE is designed to be incremental and is currently 
implemented in four catagories as Table 2 below demonstrates. 


18 Nathan, John 2003. 

19 Department of Defense 5000.1, 2003. 
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Table 4. OACE Incremental Compliances^ 


2. OACE Category 3 

Open Architecture Computing Environment category 3 describes complete 
compliance with all OACE standards to include, physical media, networks, operating 
systems, middleware and programming languages. This is critical in reuse of components 
and allows for the interoperability between different computing infrastructures. The goal 
for the full integration of Category 3 is the year 2008, with the main component being the 


20 Department of Defense, NAVSEA, PEG IWS 2004. 


17 

































standard middleware .21 Middleware is the use of software which allows interoperability 
between two different closed systems. Rather than proprietary based middleware, 
standards-based solutions will be used instead to meet the ever-changing requirements of 
the system. Standards-based solutions are those which meet the industry regulated norm, 
such as the “user-friendly” norm which Microsoft created when they released windows 
on an international level. 

E. SECURITY QUESTIONS 

One central concern of the Department of Defense with regards to the shift to OA 
and OACE is the need to maintain security in military systems. Some have speculated 
that to let open architecture be the prevailing architecture for fleet releasable software 
maintenance and upgrade would be to let the proverbial "wolf in sheep's clothing" 
penetrate our defenses. These concerns stem from the fear of malicious code causing a 
whole defense system to malfunction. Supporters of open architecture state that because 
newer systems are so open to review by so many different sources that the possibility of 
malicious code passing under the eyes of so many is slim. 

F. SERVICE ORIENTED ARCHITECTURE (SOA) SOUUTIONS 

Service oriented architecture is a permutation of OA in that it is the ultimate in the 
OA Tenets of “modularity” and “reuse.” SOA seeks to combine many different services 
that communicate through XML messages across a common web so that they can be 
interchangeably used to complete a task. It makes software based changes rather than 
hardware based changes. Table 3 is a before and after example of the implementation of 
SOA, and it has both pros and cons. 


21 Department of Defense, NAVSEA, PEG IWS 2004. 
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Table 5. Before & After SOA22 


1. SOA Benefits 

With the introduction of SOA any service that requires examining, such as a 
broken pipe or a computer that has “crashed” can be evaluated by meshing whichever 
portions of an organization that need to be meshed together to address the specific 
problem at the time of an anomaly. Some other benefits include a higher potential ROI, 
visibility or enterprise level business processes, reduced redundancy and ease of 
interoperability internally and between third parties.23 SOA could be extremely beneficial 
in this case with organizational “stove-pipes” and bridging legacy systems that otherwise 
would be non-interoperable in the environment. 


22 U.S. Army PEO lES 2006. 

23 Ibid. 
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SOA Drawbacks 


There are several drawbacks to SOA. If there is a large volume of transactions 
within the system, for instance, an online bank, then SOA would require a massive 
amount of time and dedicated resources to fully realize its potential. Additionally, 
security is as much of an issue in SOA as it is in OA. At times it may be easier to have 
tightly group interfaces with a history of collaboration, rather than loose interfaces. This 
is especially sensitive where AEGIS is concerned because if the interface is not tightly 
coupled, then any latency present could cause enough delay to give an enemy superiority 
over our assets. As an air defense platform any increase in latency would be 
unacceptable. 

G. DISTANCE SUPPORT MAINTENANCE SOLUTIONS 

According to 2006 Distance Support Policy released by the Chief of Naval 
Operations, distance support is rapidly becoming “the Fleet’s principal web-based 
readiness enabler.”24 At a minimum, the current distance support system, “combines 
people, processes, and technology into a collaborative infrastructure without regard to 
geographic location.”25 In other words, ships can be underway for several months and 
communicate with shore based sites to fix software and hardware problems that occur on 
board and, hopefully, resolve them without pulling into a foreign port or returning to the 
shipyards. 

The future of distance support will also include shore based monitoring of 
systems, in much the same way that cars sold in 2006 and 2007 can communicate with 
central databases and give a report of their technical status which is then emailed to the 
owner of the vehicle. This form of distance support, called remote monitoring and 
notification, in a possible form of procedure for shipboard operations, is displayed in 
Table 4 below. 


24 Chief of Naval Operations 2006. 

25 Ibid. 
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Table 6. Remote Monitoring and Notification 


From the table, the shipboard information is constantly monitored in the 
“data/information acquired phase” which then relays the information to the shore-side 
server for diagnostics and assessments of trends and material readiness. If there is a 
problem with the systems maintenance and risk recommendations are made, documented, 
and then analyzed in metrics; the ship is notified. If there is no detected issue then the 
monitoring cycle will repeat and send a clean report to the ship. 

When distance support is used correctly it complements the tenets of OA quite 
effectively, if upgrades and the necessary changes can be made to an open system. It 
allows for modularity and design reuse that can be distantly monitored and repaired 
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rather than requiring a visit to port for the problem to be fixed. Additionally, in the figure 
above, remote monitoring can provide automation in the process of upgrades and repairs, 
and automation in most cases, leads to a decrease in number of employees, hence, an 
increase in a systems’ Return on Investment 

H. BEST PRACTICES 

1. Distance Support 

a. Maintenance Free Operating Period (MFOP) and Acoustic 
Rapid Commercial Off-the-shelf (COTS) Insertion (ARCI) 

In 2005, Naval Sea Systems Command (NAVSEA) completed a pilot 
program to test the feasibility of a Maintenance Free Operating Period (MFOP) on the 
Acoustic Rapid COTS Insertion (ACRI) System. The ARCI system was designed to 
replace the AN/BSY-1 and the AN/BQQ-5 on the fleet’s in-service submarines 
(688/688I/Trident/Seawolf).26 ARCI was a success in its own right, in that it effectively 
demonstrated the use of OA with COTS technology on a large scale in the fleet and 
allowed for technology insertion and refreshment.27 

The ARCI MFOP program was conducted over a one year time span and it 
tested the use COTS technology and the COTS support provided to design ARCI in such 
a way to enable MFOP. Four platforms participated in the testing and over the course of 
one year no maintenance was required in any of the four. One resulting benefit of this 
test, for the purposes of this research, was that they implemented distance support 
capabilities into the ARCI system before they conducted the test.28 Most particularly, the 
following results are applicable for the formulation of “To Be” models for AEGIS 
software maintenance and upgrade: 


Lockheed Martin 2005. 

27 Ibid. 

28 NAVSEA Surface Warfare Logistics and Maintenance 2005. 
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A database of maintenance related data was built into the ARCI system 
which provides the capability to perform statistical analysis of system 
performance and improve availability. An availability correlation function 
was developed to monitor system parameters and make recovery 
recommendations to system operators... An additional benefit of the MFOP 
Pilot Program was to develop and implement functionality in the ARCI system 
which further enables the system to be supported via Distance Support 

initiatives.29 

Using the advances outlined above, the “To-Be” models were formulated and the basis 
for those changes was grounded in research that has proven its MFOP reliability over the 
course of an entire year. 


NAVSEA Surface Warfare Logistics and Maintenance 2005. 
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III. PROCESS DIAGRAMS AND “AS IS” MODELS 


A. INTRODUCTION 

Program Executive Office, Integrated Warfare Systems (PEO IWS), with its 
creation in 2002, began an initiative to implement open architecture (OA) throughout the 
Navy’s Integrated Warfare Systems. One of the current initiatives is to implement OA 
into the sustaining engineering process associated with the AEGIS system. 

Sustaining engineering in the AEGIS system was identified as an area of concern 
by a Component Business Model (CBM) analysis completed by IBM. The AEGIS 
software maintenance and upgrade process was further identified by the research team as 
an area where the most improvements could be made by implementing an open 
architecture framework. In order to accomplish the implementation of open architecture 
into the AEGIS software maintenance and upgrade process, metrics must be determined 
to discover which areas of the software maintenance and upgrade process would be the 
best candidates for open architecture. 

This proof of concept/case study will utilize information gathered from subject 
matter experts (SME’s) in both the Surface Warfare Elect and the Naval Surface Warfare 
Center (NSWC). The process information gathered from these subject matter experts 
will be utilized to provide a Return on Investment (ROI) analysis using the Knowledge 
Value Added (KVA) methodology. This will be an analysis of the “As Is” system 
configuration or the baseline case. The ROI discovered through the KVA methodology 
will be analyzed to determine if open architecture could improve the sustaining 
engineering process. A Real Options (RO) analysis will be conducted in future research 
on the “To Be” software maintenance and upgrade process model in order to provide 
PEO IWS with options and risks for future courses of action. 
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B. RESEARCH QUESTIONS 


Measure of effeetiveness (MOE) for open arehiteeture systems have been 
aceurately derived through the Knowledge Value Added Methodology. This proof of 
concept was conducted in previous research by Capt. Joseph Uchytil, USMC in his thesis, 
“Assessing the Operational Value of Situational Awareness for AEGIS and Ship Self 
Defense System (SSDS) Platforms through the Application of the Knowledge Value 
Added (KVA) Methodology.” This thesis is intended to draw on Capt. Uchytil’s proof of 
concept in order to answer the following research questions: 

* Using IBM’s Component Business Modeling (CBM), what are the areas 
of highest concern and cost in the AEGIS weapons system as they relate to sustaining 
engineering? 

* What are the “best practice” examples of sustaining engineering in the 
commercial or military environment and how do they improve the processes of system 
development and acquisition life-cycles? What are the best examples of “design for 
maintenance” systems? 

* What is the benefit of OA in the process of sustaining engineering? We 
will apply the KVA methodology to the areas of highest concern as identified by IBM in 
their CBM to address these questions. . 

* What overall numerical effect does OA have on Sustaining Engineering 
and is it appreciable and low-risk enough to system development and the DOD 
acquisition lifecycle? 

C. ANALYSIS AND DATA COLLECTION 

I. The Software Maintenance and Upgrade Process 

The AEGIS software maintenance and upgrade process is a very complex method 

encapsulating a large number of processes in four main phases. The phases are the 

requirements definition phase, the design phase, the test phase, and the implementation or 

installation phase. These are the basic phases in any software maintenance/upgrade 
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process whether commercial, government, or non-profit. Depending on the type of 
upgrade required, the severity of the problem it is intended to fix, and the timeliness in 
which it is installed, the process can be slightly tailored to produce more immediate 
results. 

One example of this process being tailored to fit a certain fleet need would be 
when an Emergent Update is required was discussed. The Emergent Update process is 
utilized for problems with software that are considered “showstoppers” or a Priority lA 
problem. Pending approval by both the project manager and the customer, the software 
maintenance and update process will be tailored to address only the Priority lA process 
and will be completed in approximately one month. Emergent Updates happen on rare 
occasions and approximately 95% of AEGIS software upgrades go through the entire 
software maintenance and upgrade process. 

The entire AEGIS software upgrade life cycle is intended to take eighteen 
months, but typically takes closer to twenty four months due to problems that are found 
during the testing phase or failure of certifications. This software maintenance and 
upgrade process involves many sub processes in each one of its main processes. These 
sub processes may, or may not, have a bearing on the rest of the processes in the software 
maintenance and upgrade process. The fact that some of these sub processes may be able 
to function in a stand alone capacity makes the analysis of the software maintenance and 
upgrade process very difficult. 

2. Data Collection Challenge 

Due to the complex nature of the software maintenance and upgrade process and 
the large number of people involved, collecting accurate data to be used by the KVA 
methodology proved to be a challenge. There were only a few SMEs who understood the 
complexity of the process. Also, outputs and learning time associated with each process 
and sub process are not documented. This is coupled with the confusion that occurs 
between learning time and time spent in a Navy training course to learn the job. The 
Navy training courses are often of a uniform length of time, no matter the complexity of 
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job and subject matter experts often confuse these training times with actual learning 
time. This leads to a slow data collection process because the data needed for the KVA 
analysis is not readily available. There is also a need to separate the time spent in a Navy 
training course or school learning the specific process and time spent learning other 
skills. Due to these concerns, data collected for this analysis was collected through 
conversations and surveys given to Subject Matter Experts (SME’s). The data was then 
aggregated to capture the AEGIS software maintenance and upgrade process as a whole. 

D. DEFINING THE AEGIS SOFTWARE MAINTENANCE AND UPDATE 
PROCESS 

1. AEGIS Software Maintenance and Update Process Overview 

The AEGIS software maintenance and update process takes place in two primary 
areas; on ship and off ship. The on ship portion of the process takes place aboard an 
AEGIS equipped U.S. Naval vessel and is conducted by Surface Warfare fleet personnel 
and various support personnel including contractors. The on ship portion of the software 
maintenance and update process deals with identifying problems that were not found in 
the testing phase of the process and also deals with installation and on ship testing of the 
fielded AEGIS software update. The off ship portion of the process takes place at Naval 
Surface Warfare Center, Dahlgren, VA. This primarily deals with the requirements 
definition phase, the design phase and the testing phase. 

2. Defined AEGIS Software Maintenance and Update Processes 

Since the AEGIS software maintenance and update process is a complex process 
involving may processes and sub processes, it was necessary to breakdown the process 
into each of its individual processes and sub processes. This breakdown allows for a 
more detailed analysis of each process contained within the software maintenance and 
update process. 

The aggregate processes associated with the AEGIS software maintenance and 
update process are depicted in Table 7. These processes were developed as a result of 
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communication with several Subject Matter Experts. Although the process could differ 
for high priority updates, such as an emergent update, the process depicted encompasses 
almost all AEGIS software maintenance and updates. Some sub processes were captured 
within their larger process to provide a level of decomposition that was sufficient to 
produce accurate results for the KVA methodology. 
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AEGIS WEAPONS SYSTEM 
SOFTWARE UPDATE 
PROCESS 
2 / 24/2007 



Table 7. AEGIS software maintenance and update process (Aggregate level) 
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E. ON SHIP PROCESSES 


The on ship process reflects all of the processes in the AEGIS software 
maintenance and update process that would take place aboard a U.S. Naval vessel. The 
detection of equipment and software failures and the effect of these failures on the 
mission capability of the AEGIS system are made by two departments. The anomaly 
identification and repair function works with the detection, reporting and resolution of 
software anomalies function. Casualty Report (CASREP) procedures, under normal ship 
operations, are explained below. 

1. Software Anomaly Detected 

Software anomalies can be detected through many methods. In most cases the 
software and test engineers in the test phase of software maintenance and update process 
detect these anomalies. Software anomalies can also be detected during the combat 
systems integration testing after shipboard delivery. This is the first time that the 
software is in its shipboard configuration and allows for fully fielded software updates to 
be operationally tested. On rare occasions software anomalies can also be detected 
during normal shipboard operation by Surface Warfare fleet operators. 

2. Cause of Anomaly Determined 

Personnel who have observed the anomaly when the software is in its shipboard 
configuration attempt to collect data regarding the anomaly and also attempt to trace the 
anomaly to its source. 

3. Software Bug Report Submitted 

All data and information surrounding the anomaly and its source are collected. 
The configuration and status of the AEGIS system as well as any environmental data is 
also gathered and reported to the Program Manager. 
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F. OFF SHIP AGGREGATE PROCESSES 

The following represents an aggregate software maintenance and update process 
procedure taken at the NSWC, Dahlgren, V.A. to process software anomalies when an 
anomaly has been detected on software in its shipboard configuration. The aggregate 
process executed at NSWC, Dahlgren, V.A. will be further decomposed and explained in 
the next section. 

1. Software Anomaly Verified 

The Project Manager and the project team work to recreate the conditions in a lab 
that were documented in the Software Bug Report in an effort to recreate the software 
anomaly. If the software anomaly is verified the software maintenance and update 
process will continue. 

2. Anomaly Appended to Working List of Known Issues 

The software anomaly is documented in the Computer Program Change Request 
(CPCR). Software anomalies are tracked in the ACCESS/STARSY database. The CPCR 
is also assessed by the Joint Change Review Board (JCRB) and the Software 
Configuration Change Board (SCCB) for inclusion in the baseline. 

3. Workaround Developed 

If CPCR is not corrected, then if a workaround exists to allow for avoidance of 
the anomaly, it is documented in the database and included in the Computer Program 
Design Document (CPDD)/Crew Brief. 

4. New Software Version Developed 

Teams of software programmers develop new versions of the software which not 
only serve to fix anomalies, but also implement new functionality and exploit new 
technologies. 
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5. 


Directed Software Anomalies are Resolved 


Anomalies that are evident in the new software and are found through 
certification testing in labs are resolved and the required changes are made to the new 
software. 


6. New Software Fielded to Units 

Teams of contractors and support personnel arrive aboard ship to deliver new 
software, install the new software and conduct crew briefs and training. 

G. OFF SHIP PROCESS DECOMPOSITION 

After the initial process model depicted above in Table 7 was developed, it 
became apparent after numerous conversations with subject matter experts that the most 
significant improvements to the software maintenance and update process would result in 
restructuring the Off Ship component. Due to this, a higher level of decomposition was 
needed on for the Off Ship processes occurring at NSWC, Dahlgren V.A. Subject matter 
experts were again consulted and a more detailed process model for the AEGIS software 
maintenance and update process that occurred at NSWC, Dahlgren V.A. was developed. 
The more detailed Off Ship AEGIS software maintenance and update process is depicted 
in Table 8 and explained below. 
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> Shipboard Delh/ery 


Combat System 
integration Test 


Color Co<te 

Requirements Definition Phase • Green 
Design Phase - Red 
Test Phase-Blue 

Implemeniation and Instalation Phase - Yellow 


Table 8. AEGIS software maintenance and update process (Off Ship) 
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1. 


Inputs and Requirements Gathered 


Fleet inputs, external interface requirements and new system requirements are all 
gathered during this process. 

2. System Design Review 

This is a scheduled review that is technical in nature. “This review shall be 
conducted to evaluate the optimization, correlation, completeness, and risks associated 
with the allocated technical requirements. Also included is a summary review of the 
system engineering process which produced the allocated technical requirements of the 
engineering planning for the next phase of effort.”30 

3. ECPs, SCPs, ICRs 

This process documents any changes that need to be made to the software after it 
has undergone the system design review. These will be documented through an 
Engineering Change Proposal (ECP), Software Change Proposal (SCP) and an Interface 
Change Request (ICR). 

4. Approval Process 

The change proposals and requests are then sent through an approval process 
where the SCP awaits Software Configuration Change Board (SCCB) approval, and the 
ICR undergoes Integration Working Group (IWG) approval. The aggregate approvals are 
then sent to PMS 442 for approval. PMS 442 is the acquisition organization responsible 
for that product, in this case it would be NAVSEA program management. 


30 Department of Defense 1985. 
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5. Design Review 


This is the first step in the design phase of the AEGIS software maintenance and 
update process. The design review process includes a preliminary design review (PDR), 
and a Critical Design Review (CDR). 

6. Design Walkthrough 

The design walkthrough process includes writing and inspecting code, unit test 
and analysis and the debugging of code. 

7. PDS/SDD 

The previous design walkthrough process produces the Preliminary Design 
Specification (PDS) and the Software Design Document (SDD) 

8. Develop Test Plan 

This is the first process in the test phase of the software maintenance and update 

process. In this process a test plan is developed using test specifications and the test case 
design process. For the purposes of this thesis, it is not necessary to describe the test case 
design process. 


9. Test Procedures 

The test procedures are the output of the develop test plan process. These 
procedures will later be utilized in the test execution and data analysis process. 

10. Test Readiness Review Process 

The test plan and test procedures are reviewed in order to make sure that the test 
process will be most effective. In order to achieve maximum efficiency, collaborative 
testing and data analysis are included. 
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11. Identify/Resolve Issues 


This process includes an assessment of any CPCRs for possible program update 
and also the certification impact of any CPCRs. 

12. Certification Impact Decided 

If any of the CPCRs are determined to have the potential for a high certification 
impact then the program must be updated before it can be sent to the test execution and 
data analysis portion of the software maintenance and update process. If the CPCRs are 
not determined to have a high certification impact the software, then is sent directly to 
test execution and data analysis. 

13. Update Program (High Certification Impact) 

For software that contains CPCRs that are determined to have a high certification 
impact, the software must be updated. This includes another unit test and analysis and an 
assessment of the certification retest. 

14. Updated Program 

Once the program is updated and the unit test and analysis and an assessment of 
the certification retest has been conducted, the software is then sent to test execution and 
data analysis. 

15. Test Execution and Data Analysis 

The software is tested in lab conditions in order to detect any potential problems 
that might arise before the software is fully fielded. It consists of three sub processes. 
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a. Software Anomaly Discovered 

A software anomaly is found under lab conditions. 

b. Anomaly Documented in CPCR Database 

A CPCR is generated for the anomaly and is then entered in the 
ACCESS/STARSY database. 

c. CPCR Assessed 

The CPCR is prioritized and its certification impact is assessed. The 
CPCRs operational impact is also assessed and, if possible, a workaround is established. 

16. Document Results 

The results from the test execution and data analysis portion of the software 
maintenance and update process are documented. 

17. Conduct Functional Area Assessment 

This process is an analysis conducted at a higher level in order to prepare the 
software for the certification panel review. 

18. Conduct Certification Panel 

A certification panel assesses the software’s results from the test execution and data 
analysis process and certifies the software for fielding. 

19. PAT/FQT 

A Preliminary Acceptance Test (PAT) and Functional Quality Testing (FQT) is 
then conducted on the software. 
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20. Data Analysis 


The data collected from the PAT and the FQT is assessed and analyzed in 
preparation for the Lab Combat Systems Integration Test. 

21. Lab Combat System Integration Test 

This process includes any final testing that occurs in the lab environment 
including any software trouble reports (STR) that are collected. CPSA Analysis and a 
CPSA report. 

22. Shipboard Delivery 

The new software is fielded to operational units and installed by teams of 
contractors and support personnel. Crew briefs and training are also conducted 

23. Shipboard Combat System Integration Test 

The software is tested in its fully fielded shipboard configuration. Due to the fact 
that this is the first time the software is in the shipboard configuration it must be tested 
for functionality. This also ensures that it is interoperable with all combat systems already 
in place on the ship. 

H. “AS IS” KVA ANALYSIS - ON SHIP AND OFF SHIP AGGREGATE 

PROCESS MODEL 

An analysis of each sub process in the AEGIS software maintenance and upgrade 
process for the On Ship and Off Ship aggregate process model is provided in the 
following tables. The information provided for each analysis was produced through 
discussions with subject matter experts. Each category for the KVA analysis is defined 
below. 
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1 . 


Title of Head Process Executer 


The “Title of Head Process Executer” category represents the job title of the 
person executing or overseeing the execution of the specific process or sub process. The 
process executors pay grade is indicated next to their job title. For purposes of this thesis, 
we used pay grades that erred on the high side to be conservative. If several executors 
with different pay grades were executing the same process then the highest pay grade was 
used as a baseline for that process executor. This produces the most conservative KVA 
results. Some other basic assumptions for this category were: 

* Each pay grade was to be at a Step 6 level within their respective pay 
grade 

* Base pay was location adjusted for San Diego, CA as a baseline 

* Civilian software contractor salary market comparables were estimated to 
be 175% of government salary 

2. Number of Employees 

The “Number of Employees” category represents the number of government 
employees or contractors which are involved in the specific sub process. If more than 
one person was involved in both the parent and specific sub processes, that person is 
documented separately for each sub process. 

3. Rank Order of Difficulty 

An ordinal ranking of the relative difficulty of learning each of the processes is 
collected and used to ensure that the “Relative Eearning Time” and “Actual Average 
Training Period” estimates are reliable. By allowing the subject matter experts to rank 
each of the sub processes (1 being the least complex) outside of the context of time units 
a correlation can be made between the “Rank Order of Difficulty” and the “Relative 
Beaming Time.” If a correlation of 80% is achieved the results appear to be reliable and 
the “Relative Eearning Time” can be considered an accurate description of the relative 
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difficulty of the sub processes. If a correlation of 80% is not achieved, the results must 
be closely scrutinized and the subject matter experts must be resurveyed and possibly 
given a more in-depth explanation of the concept of “Relative Learning Time.” 

4. Relative Learning Time 

The “Relative Learning Time” category represents a distributed relative amount 
of 100 hours of learning time among the processes. “Relative Learning Time” assumes 
an “average person” will learn all he/she needs to know to successfully complete all the 
tasks in each process. This learning time estimate includes the time it would take to learn 
how to produce the same output that any automation (e.g. information systems) currently 
produces. The 100-hour learning period is distributed according to how difficult and 
complex the processes are for the “average person” to learn. The purpose is to determine 
“Relative Learning Times” for each process given the 100-hour total. This helps identify 
the most complex processes and can be used as another internal reliability measure. 

5. Actual Average Training Period 

The “Actual Average Learning Time” is what the actual average training time in 
hours is for the “average person” for each process. This would be for a new employee 
with no background who would be required to learn everything to produce the outputs of 
the given processes. Learning time includes both formal training and on the job training. 

The results from “Relative Learning Time” and “Actual Average Learning Time” 
are also correlated. If a correlation of 80% is achieved the results appear to be reliable 
and the “Actual Average Learning Time” can be considered an accurate description of 
the “Relative Learning Time” of the sub processes. If a correlation of 80% is not 
achieved, the results must be closely scrutinized and the subject matter experts must be 
resurveyed and possibly given a more in depth explanation of the concept of “Relative 
Learning Time” along with “Actual Average Learning Time.” In some cases, subject 
matter experts may associate “Actual Average Learning Time” with a school or training 
period associated with the process. These schools and training periods are generally 
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conducted over a uniform length of time and do not accurately reflect the “Actual 
Average Learning Time.” Assumptions include: 

* On the job training is estimated to be 50% of the time learning the task 
and 50% of the time actually performing the task. 

* On the job training was conducted 8 hours a day, 5 days a week, and 50 
weeks per year. 

* On the job training occurs over a one year period. 

6. Percentage Automation 

Each sub process has a “Percentage Automation” associated with it between 0 and 
100. This number captures the knowledge that is embedded in any information 
technology so that it can be accounted for in later calculations. This number represents 
the percentage of information technology that it utilized so that a process executor would 
not have to accomplish the task. For example, a process that has 100% automation would 
not require any process executors and would be accomplished fully by the automation 
tools listed for that process. If a process has 0% automation, no automation tools are 
utilized and the process is totally executed by the process executors. These numbers are 
estimates based on subject matter expert’s observations and experience. One basic 
assumption associated with this: 

* “Replacement Technology,” is automation that will reduce the number of 
process executors associated with the process without increasing the output of the 
process. 

7. Times Performed in a Year 

The “Times Performed in a Year” category represents the number of times each 
sub process is acted upon by a head process executor in a given year. The values were 
obtained by asking subject matter experts for their inputs to determine a valid estimate for 
the year long period. 
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8. Average Time to Complete 


Each time a sub process is acted upon (as indicated in the “Times Performed in a 
Year” category) there is a specific amount of time that it takes for each sub process to be 
satisfactorily completed. This category represents the number of hours it takes a person 
trained in each process/sub process to complete each task. 

9. Automation Tools 

The “Automation Tools” category represents any tools such as MS Office, Visio, 
SIPR Database or Belvoir Paperless Office. This is used as a baseline for any automation 
tools that are already in use for the process and may provide insight for the 
implementation of other automation tools. 

10. Total Learning Time (TLT) 

This category is produced by dividing the “Actual Average Learning Time” by 
the “Percent Automation.” Because we assume “replacement technology,” the formula 
used to determine TLT is “Actual Average Learning Time’7(l-“Percent Automation”). 
This provides a total time, in hours, for each process to include the learning time that is 
present in the automation tools. Lor example, if it takes one hour to learn a system that is 
50% automated then the total learning time associated with the process is two hours, one 
hour associated with the process executors and one hour associated with the automation 
tools. 


11. Total Knowledge 

This category is a representation, in hours, of all of the knowledge for each 
process that occurs over the one year time frame in which the survey encompasses. The 
“Number of Employees” category is multiplied by the “Times Performed in a Year,” and 
the “Total Learning Time” categories. 
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12. Personnel Cost 


This number represents the costs that are associated with the government 
employees associated with each process. This number is calculated by multiplying the 
employee’s hourly wage with the “Average Time to Complete,” the “Number of 
Employees,” and the “Times Preformed in a Year” categories. This number shows the 
cost of process only associated with personnel over the course of a year. Even though 
personnel may be involved with other tasks during their employment, this number shows 
the wages paid only based on their work with the specified process. Assumptions 
include: 

* An average employee works 250 days per year 

* Wages are adjusted for the location of San Diego, C.A. 

13. Other Costs 

The “Other Costs” category represents the cost of the process executors utilizing 
workstations that can access the Navy Marine Corps Intranet (NMCI). This category was 
calculated by taking the average cost per seat associated with NMCI and multiplying it by 
the “Number of Employees” category. Assumptions include: 

* NMCI cost per seat is approximately $300031 

* Assumes a “Red Seat” - Pentium 800MHz. This provides 
performance for use with 2-D and light 3-D graphics or 
engineering related applications, applications that require 
additional processing capability. 32 


31 Navy Marine Corps Intranet 

32 Navy Marine Corps Intranet 
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14. Total Costs 


The “Total Costs” category represents the total costs of the process. This 
category was calculated by adding the “Personnel Costs” category with the “Other Costs” 
category. 

15. Price 

This category represents the price that it would cost if the process was executed 
by civilian contractors rather than government employees. The “Price” category was 
calculated much in the same way that the “Personnel Cost” category was calculated, 
except using contractor hourly wages in lieu of government employee hourly wages. 
This category was calculated by multiplying the contractor wage per hour by the 
“Number of Employees,” the “Average Time to Complete,” and the “Times Performed in 
a Year” categories. Assumptions include: 

* Civilian software contractor market comparables were estimated to be 
175% of government salary 

16. Denominator 

This category shows the cost associated with producing the output of each 
process. It is the same as the “Total Costs” category. 

17. Numerator 

The “Numerator” category is the “percentage of the revenue or sales dollar 
allocated to the amount of knowledge required to obtain the outputs of a given process in 
proportion to the total amount of knowledge required to generate the corporation’s 
salable outputs.”^3 For the purposes of this thesis, the revenue allocated to the amount of 
knowledge can be compared to the amount of knowledge that is present in each process 
or sub process. This can also be thought of as the total knowledge multiplied by the price 


33 Housel, Thomas 2001; 45. 
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of each common unit. This value was calculated by first finding the price of each 
common unit. The price per common unit was calculated by dividing the total 
knowledge into the total price. The formula is (price of the entire process)/(total 
knowledge of the entire process). The “Numerator” was then calculated by multiplying 
the total knowledge associated with each sub process with the price of each common unit. 
Establishing a price per common unit is important when developing the to-be model 
which will be discussed in the Chapter IV. 

I. ROK 

With each process or sub process there is both a cost and a revenue associated 
with producing an output. The Return on Knowledge (ROK) provides a representation of 
how well the assets within a process are distributed in relation to one another by utilizing 
the costs and revenues associated with each sub process. The ROK is calculated by 
dividing the “Numerator” by the “Denominator.” ROK’s can be compared within a 
process to determine which processes are utilizing assets in an efficient manner and 
which processes need to be changed, perhaps by the utilization of automation tools, in 
order to improve efficiency. Although ROK is a very valuable tool, a low ROK does not 
dictate that a process is in need of increased automation, but serves as an indicator that 
the process should be analyzed more closely to discover if process efficiency can be 
improved. 

J. ROI 

“ROI” or return on investment is a common accounting term that is widely 
understood by the financial community. For this reason, it is a slightly more meaningful 
number than “ROK.” Essentially, it is a very similar number to “ROK,” just a different 
unit of measure. In financial terms, “ROI” is the profit or loss resulting from an 
investment transaction, usually expressed as an annual percentage return. “ROI” is a 
return ratio that compares the net benefits of a project verses its total costs. In financial 
terms, “ROI” is calculated by profit minus investment all divided by investment. For the 
purposes of KVA, “ROI” is calculated by the “Numerator” minus the “Denominator” all 
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dived by “Denominator.” Much like “ROK’s,” “ROI’s” can be compared within a 
process to determine which processes are utilizing assets in an efficient manner and 
which processes need to be changed, perhaps by the utilization of automation tools, in 
order to improve efficiency. 

K. “AS IS” PROCESS DATA - ON SHIP AND OFF SHIP AGGREGATE 

PROCESS MODEL 

Each of the AEGIS software maintenance and upgrade processes and sub 
processes will be presented for evaluation. The ROKs and ROIs that were calculated for 
each process and sub process will be utilized to find the differences in efficiency for each 
of the processes and sub processes. The analysis of the “As Is” process data will provide 
insight on the amount and location of knowledge assets throughout the AEGIS software 
maintenance and upgrade process. 

I. On and Off Ship Aggregate Process 

Table 9 and 10 depict the on and off ship aggregate processes included in the 
KVA analysis for the AEGIS software maintenance and upgrade process that occurs for 
one AEGIS ship. 
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Process 

Title of Head Process Executer 

Number of Employees 

Rank Order of Difficulty 

Relative Learning 
Time 

Actual Average 
Learning Time 

Percentage 

Automation 

Times Performed In a 
Year 

Average Time to 
Complete 

Automation Tools 

Software Anomaly Detected 

Proiect Manager [GS-111 

2 

4 

10 

300 

0% 

10 

20 

N/A 

Cause of Anomaly Determined 

Technology Director IGS-12/131 

5 

8 

20 

500 

30% 

25 

40 

Advanced Software 

Software Bug Report Submitted 

Project Manager (GS-11) 

1 

3 

10 

300 

10% 

10 

4 

MS Word 

Anomaly Verified 

Project Manager (GS-11/12) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly Appended to Working List of Known Issues 

Fleet Support (GS-9/101 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround Developed 

Project Manager (GS-11/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New Software Version Developed 

Lead Programmer (GS-13/141 

10 

9 

20 

640 

20% 

2 

200 

Compiler 

Known Anomalies are Resolved 

Lead Programmer (GS-12/131 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New Software Version Fielded to Units 

Fleet Support (GS-9/10) 

3 

1 

5 

160 

10% 

20 

80 

Tracking 


Correlation (Ordinal to RLT) 0.877032523 

Correlation (RLT to AATP) 0.845793835 


Process (continued) 

TLT 

Total Knowledge 

Personal Cost 

Other Costs 

Total Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

Software Anomaly Detected 

300 

6000 

$ «,701.00 

$ 600.00 

$ 14,301.00 

$ 23,976.75 

14301 

56826.27304 

397% 

297% 

Cause of Anomaly Determined 

714 

89286 

$ 244,096.88 

$ 7,500.00 

$ 251,596.88 

$ 427,169.53 

251596.875 

845629.0632 

336% 

236% 

Software Bug Report Submitted 

333 

3333 

$ 370.10 

$ 60.00 

$ 1,430.10 

$ 2,397.68 

1430.1 

31570.15169 

2208% 

2108% 

Anomaly Verified 

300 

6000 

$ 16,421.50 

$ 600.00 

$ 17,021.50 

$ 28,737.63 

17021.5 

56826.27304 

334% 

234% 

Anomaly Appended to Working List of Known Issues 

178 

1778 

$ 1,247.00 

$ 60.00 

$ 1,307.00 

$ 2,182.25 

1307 

16837.41424 

1288% 

1188% 

Workaround Developed 

160 

3200 

$ 16,421.50 

$ 600.00 

$ 17,021.50 

$ 28,737.63 

17021.5 

30307.34562 

178% 

78% 

New Software Version Developed 

800 

16000 

$ 230,750.00 

$ 6,000.00 

$ 236,750.00 

$ 403,812.50 

236750 

151536.7281 

64% 

■36% 

Known Anomalies are Resolved 

625 

6250 

$ 97,638.75 

$ 3,000.00 

$ 100,638.75 

$ 170,867.81 

100638.75 

59194.03442 

59% 

■41% 

New Software Version Fielded to Units 

178 

10667 

$ 149,640.00 

$ 7,200.00 

$ 156,840.00 

$ 261,870.00 

156840 

101024.4854 

64% 

■36% 

Totals 


142513 

1 771,286.73 

1 25520.00 

1 796906.73 

1 1.349751.77 

796906,725 

1349751.769 

169% 

55 % 


Personal Costs 

Base Pay 



Location Adjusted 


Hourly Wage Contractor Wage 

GS9 

$ 

45,294.00 

$ 

56,617.50 

$ 

28.3t $ 

49.54 

GS10 

$ 

49,880.00 

$ 

62,350.00 

$ 

31.18 $ 

54.56 

GS11 

$ 

54,804.00 

$ 

68,505.00 

$ 

34.25 $ 

59.94 

GS12 

$ 

65,686.00 

$ 

82,107.50 

$ 

41.05 $ 

71.84 

GS13 

$ 

78,111.00 

$ 

97,638.75 

$ 

48.82 $ 

85.43 

GS14 

$ 

92,300.00 

$ 

115,375.00 

$ 

57.69 $ 

100.95 


Price Per Common Unit 


$ 

9.47 





Table 9. Off Ship Aggregate Process (One ship) 
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2 . 


On and Off Ship Aggregate Process for Entire AEGIS Fleet 


Table 10 depicts the further decomposed off ship process included in the KVA 
analysis for the AEGIS software maintenance and upgrade process that is scaled to 
include all AEGIS ships in the U.S. Elect. Assumptions for scaling the process data 
include: 

* Updates are delivered to each ship twice a year. 

* Each update contains five anomaly fixes. 

* The “New Software Version Eielded to Units” occurs 168 times per year 
based on 84 AEGIS ships and two updates per year. 
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Process 

Title of Head Process Executer 

Number of Employees 

Rank Order of Difficulty 

Relative Learning 
Time 

Actual Average 
Learning Time 

Percentage 

Automation 

Times Performed In a 
Year 

Average Time to 
Complete 

Automation Tools 

Software Anomaly Detected 

Project Manager (GS'11) 

2 

4 

10 

300 

0% 

10 

20 

N/A 

Cause of Anomaly Determined 

Technology Director (GS-12/13) 

5 

8 

20 

500 

30% 

25 

40 

Advanced Software 

Software Bug Report Submitted 

Project Manager (GS-11) 

1 

3 

10 

300 

10% 

10 

4 

MS Word 

Anomaly Verified 

Project Manager (GS-tt/tS) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly Appended to Working List of Known Issues 

Fleet Support (GS-9/1(l) 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround Developed 

Proiect Manager (GS-11/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New Software Version Developed 

Lead Programmer (GS-13/14) 

10 

9 

20 

640 

20% 

2 

200 

Compiler 

Known Anomalies are Resolved 

Lead Programmer (GS-12/13) 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New Software Version Fielded to Units 

Fleet Support (GS-9/10) 

3 

1 

5 

160 

10% 

20 

80 

Tracking 


Correlation (Ordinal to RLT) 0.877032523 

Correlation (RLT to AATP) 0.845793835 


Process (continued) 

TLT 

Total Knowledge 

Personal Cost 

Other Costs 

Total Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

Software Anomaly Detected 

300 

6000 

$ 13,701.00 

$ 600.00 

$ 

14,301.00 

$ 23,976.75 

14301 

4773406.936 

33378% 

33278% 

Cause of Anomaly Determined 

714 

89286 

$ 244,096.88 

$ 7,500.00 

$ 

251,596.88 

$ 427,169.53 

251596.875 

71032841.31 

28233% 

28133% 

Software Bug Report Submitted 

333 

3333 

$ 1,370.10 

$ 60.00 

$ 

1,430.10 

$ 2,397.68 

1430.1 

2651892.742 

185434% 

185334% 

Anomaly Verified 

300 

6000 

$ 16,421.50 

$ 600.00 

$ 

17,021.50 

$ 28,737.63 

17021.5 

4773406.936 

28043% 

27943% 

Anomaly Appended to Working List of Known Issues 

178 

1778 

$ 1,247.00 

$ 60.00 

$ 

1,307.00 

$ 2,182.25 

1307 

1414342.796 

108213% 

108113% 

Workaround Developed 

160 

3200 

$ 16,421.50 

$ 600.00 

$ 

17,021.50 

$ 28,737.63 

17021.5 

2545817.032 

14956% 

14856% 

New Software Version Developed 

800 

16000 

$ 230,750.00 

$ 6,000.00 

$ 

236,750.00 

$ 403,812.50 

236750 

12729085.16 

5377% 

5277% 

Known Anomalies are Resolved 

625 

6250 

$ 97,638.75 

$ 3,000.00 

$ 

100,638.75 

$ 170,867.81 

100638.75 

4972298.891 

4941% 

4841% 

New Software Version Fielded to Units 

178 

10667 

$ 149,640.00 

$ 7,200.00 

$ 

26,349,120.00 

$ 261,870.00 

26349120 

8486056.775 

32% 

■ 68 % 

Totals 


142513 

T 771,286.73 

1 25 620.00 

$ 26,989,186.73 

1 1,349T51.77 

26989186.73 

113379148.6 

420% 

315 % 


Personel Costs 

Base Pay 



Location Adjusted 


Hourly Wage Contractor Wage 

GS9 

$ 

45,294.00 

$ 

56,617.50 

$ 

28.31 $ 

49.54 

GS10 

$ 

49,880.00 

$ 

62,350.00 

$ 

31.18 $ 

54.56 

GS11 

$ 

54,804.00 

$ 

68,505.00 

$ 

34.25 $ 

59.94 

GS12 

$ 

65,686.00 

$ 

82,107.50 

$ 

41.05 $ 

71.84 

GS13 

$ 

78,111.00 

$ 

97,638.75 

$ 

48.82 $ 

85.43 

GSM 

$ 

92,300.00 

$ 

115,375.00 

$ 

57.69 $ 

100.95 


Price Per Common Unit 


$ 

9.47 





Table 10. Off Ship Aggregate Proeess (All Ships) 
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3. Off Ship Process Decomposition 


Table 11 depicts the further decomposed off ship process included in the KVA 
analysis for the AEGIS software maintenance and upgrade process that occurs for one 
AEGIS ship. 
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Process 

Title of Head Process Executer 

Number of Employees 

Rank Order of Difficulty 

Relative Learning 
Time 

Actual Average 
Learning Time 

Percentage 

Automation 

Times Performed In a 
Year 

Average Time to 
Complete 

Automation Tools 

Cost of IT 

Fleet Inputs/Extemal Interface Requirements/New System Resources 

Program Manager (ND-IV) 

15 

2 

1 

500 

15% 

3 

335 

Databases to ID issues 
for inclusion/fleet 
issues/etc 

10000 

System Design Review 

Systems Engineer (ND-IV) 

50 

4 

2 

500 

5% 

1 

335 

database/Adjudication 

10000 

ECP's/SCP’s/ICR's 

Systems Engineer (NL)-IV) 

25 






60 

ACSIS 

bUUOO 

Approval Process (Including FPRG, SCCB, IWG, PMS 422) 

PM, PMS 422, LSEA(ND-V) 

50 

8 

8 

835 

15% 

50 


ACSIS 

45000 

Design Review (Including PDR, DDWS, CDR) 

PM (ND-IV) 

50 

5 

5 

500 

5% 

3 

120 

Comment 

database/Adjudication 

tracker 

10000 

Design Walkthrough (Including Writing and Inspecting Code, Unit Test and Analysis, Debugging Code) 

Uevloper (ND-IV) 

8 

7 

7 

1000 

15% 

5(5 

80 

MS Word 

0 

Develop Test Plan (Including Test Specifications and Test Case Design) 

1 est Engineer (ND-IV) 

2 

12 

10 

1000 

5% 

50 

1000 

MS Word 

0 

Test Procedures 

lest Engineer (NU-iv) 

2 

13 

10 

1000 

5% 

bO 

1UUU 

MS Word 

0 

Test Readiness Review 

PM (ND-IV) 

30 

3 

1 

500 

5% 

3 

16 

MS PowerPoint 

0 

st Execution and Data Analysis (Software anomaly detected, anomaly documented in CPCR, CPCR assess 

Engineers (ND-IV) 

50 

14 

10 

1000 

0% 

50 

665 

1 est Director and 

TERMS are used as 
respository 

0 

Document Results 

Engineers (ND-IV) 

25 

6 

5 

500 

10% 

50 

665 

1 est Director and 

TERMS are used as 
respository 

0 

Identify/Resolve Issues (Including assessing CPCR for possible program update and certification impact) 

Engineers (NU-iv) 

25 

15 

10 

nroc 

0% 

bO 

40 

ACSIS 

50000 

High Cert Impact (Yes/No branch. No 95% of time) 

Engineers (NU-IV) 

30 

10 

10 

1000 

10% 

50 


ACSIS 

50000 

Conduct Functional Area Assessment (NO BRANCH) 

lest IPI Lead (ND-IV) 

15 

11 

10 

1000 

0% 

50 

120 

MS Excel 

0 

Conduct Certification Panel (NO BRANCH) 

PM (NU-V) 

30 

9 

10 

1000 

0% 

1 

16 

Mb PowerPoint 

0 

Correlation (Ordinal to RLT) 0.930399929 

100 



Correlation (RLT to AATP) 0.931962825 


Process (continued) 

TLT 

Total Knowledge 

Personel Cost 

Other Costs 

Total Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

Fleet Inputs/External Interface Requirements/New System Resources 

588 

26471 

$ 

729,385.03 


32,612.50 


761,997.53 

$ 1.276.423.80 

$ 761,997.53 


647,266.17 

85% 

-15% 

System Design Review 

526 

26316 

% 

810,427.81 


35,125.00 


845,552.81 


1,418,248.67 


845,552.81 


643,480.99 

76% 

-24% 

ECP's/SCP’s/ICR's 

263 

164474 

% 

1,814,390.63 


106,250.00 


1,920,640.63 


3,175,183.59 


1,920,640.63 

$ 4,021,756.20 

209% 

109% 

Approval Process (Including FPRG, SCCB, IWG, PMS 422) 

9B2 

2455882 

$ 1,357,050.00 


75,000.00 


1,432,050.00 


2,374,837.50 

$ 1,432,050.00 


60,051,917.22 

4193% 

4093% 

Design Review (Including PDR, DDWS, CDR) 

526 

78947 

$ 870,907.50 


37,000.00 


907,907.50 


1,524,088.13 

$ 907,907.50 


1,930,442.97 

213% 

113% 

Design Walkthrough (Including Writing and Inspecting Code, Unit Test and Analysis, Debugging Code) 

1176 

470588 

% 

1,548,280.00 


48,000.00 


1,596,280.00 


2,709,490.00 


1,596,280.00 


11,506,954.20 

721% 

621% 

Develop Test Plan (Including Test Specifications and Test Case Design) 

1053 

105263 

% 

4,838,375.00 


150,000.00 


4,988,375.00 


8,467,156.25 


4,988,375.00 


2,573,923.97 

52% 

-48% 

Test Procedures 

1053 

105263 

% 

4,838,375.00 


150,000.00 


4,988,375.00 


8,467,156.25 


4,988,375.00 


2,573,923.97 

52% 

-48% 

Test Readiness Review 

526 

47368 

$ 69,672.60 


2,160.00 


71,832.60 


121,927.05 


71,832.60 


1,158,265.78 

1612% 

1512% 

St Execution and Data Analysis (Software anomaly detected, anomaly documented in CPCR, CPCR assess 

1000 

2500000 

$ 

80,437,984.38 


2,493,750.00 


82,931,734.38 


140,766,472.66 


82,931,734.38 

$ 61,130,694.18 

74% 

-26% 

Document Results 

556 

694444 

i 

40,218,992.19 

$ 1,246,875.00 


41,465,867.19 


70,383,236.33 

S 41,465,867.19 

$ 16.980.748.38 

41% 

-59% 

Identify/Resolve Issues (Including assessing CPCR for possible program update and certification Impact) 

1000 

1250000 

$ 

2,419,187.50 


125,000.00 


2,544,187.50 


4,233,578.13 


2,544,187.50 


30,565,347.09 

1201% 

1101% 

High Cert Impact (Yes/No branch, No 95% of time) 

1111 

1666667 

$ 580,605.00 


68,000.00 


648,605.00 


1,016,058.75 


648,605.00 


40,753,796.12 

6283% 

6183% 

Conduct Functional Area Assessment (NO BRANCH) 

1UUU 

750000 

1 4,3b4.b3/.b0 


135,000.00 


4,489,537.50 


7,620,440.63 

b 4,48b,b3/.bU 

J 1b,339.208.2b 

408% 

3UB7o 

Conduct Certification Panel (NO BRANCH) 

1000 

30000 

$ 32.569.20 


720.00 


33,289.20 


56,996.10 

$ 33,289.20 

$ 733.568.33 

2204% 

2104% 

Totals 


10571684 

1 144,920,759.55 


4,795,492.55 



2b3,b11,293.83 

b 149,b2b,231.83 

1 253,611,295.83 
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Personel Costs 

Base Pay 



Location Adjusted 

Hourly Wage 

Contractor Wage 

ND-III 

$ 

55,585.00 

$ 

69,481.25 $ 

34.74 

$ 

60.80 

ND-IV 

$ 

77,414.00 

$ 

96,767.50 $ 

48.38 

$ 

84.67 

ND-V 

$ 

108,564.00 

$ 

135,705.00 $ 

67.85 

$ 

118.74 


Price Per Common Unit 


$ 

24.45 





Table 11. Decomposed Off Ship Process (One Ship) 
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IV. TO-BE RESULTS 


A. “TO BE” KVA ANALYSIS - ON SHIP AND OFF SHIP AGGREGATE 
PROCESS MODEL 

The “To Be” analysis is a hypothetical improved process model of the possible 
effects of a future Open Architecture (OA) enabled AEGIS software maintenance and 
upgrade process. The processes and sub processes that have been identified as having the 
potential for using OA and Distance Support have been modified to reflect the 
improvements. The potential improvements are described in detail in the following 
sections. The OA enabled processes and sub processes were developed using current 
distance support policy, Maintenance Free Operating Period (MFOP) research, and 
suggestions from subject matter experts (SMEs). 

B. OPEN ARCHITECTURE REENGINEERING 

The Naval Surface Warfare Center (NWSC) Port Hueneme, CA has been 
developing the concept of remote maintenance of Cooperative Engagement Capability 
(CEC) on ships. Through the implementation of an OA based system, the concept of 
remote maintenance could greatly improve the AEGIS software maintenance and upgrade 
process. 

Remote maintenance enables the Navy to provide distance support with 
fielded systems and enhances the ability to meet requirements of reduced 
manpower, lower cost, and faster more efficient troubleshooting and 
repair. It works within the initiatives to return ships to the Strike Group 
faster and provides safe, effective and affordable combat systems into 
future designs.34 

The AEGIS weapons system is already equipped with a link to provide distance 
support. “Currently installed as a fielded Ship Alteration (SHIPAET), Operational 
Readiness Test System Tech Assist Remote Support (ORTSTARS) allows AEGIS ships 
to establish a secure link between the Operational Readiness Test System (ORTS) and 

34 NAVSURFWARCENDIV PORT HUENEME CA 2006. 
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any shore facility equipped with SIPERNET.”35 This concept was used in the 
development of the “To Be” model for the AEGIS software maintenance and upgrade 
process. 

The Undersea Warfare community has also provided valuable assistance through 
their research with MEOP concept that has been incorporated in the Acoustic Rapid 
COTS Insertion (ARCI) program. The ARCI program incorporated hardware, software 
and COTS logistics support and technology to achieve a MEOP for 90 days. “An 
additional benefit of the MEOP Pilot Program was to develop and implement 
functionality in the ARCI system which further enables the system to be supported via 
Distance Support initiatives.”3^ These important principals helped guide the 
reengineering of the “To Be” model for the AEGIS software maintenance and upgrade 
process. 

Through an analysis of the “As Is” process data, it was determined that the most 
advantageous improvements to the AEGIS software maintenance and upgrade process 
could be achieved through implementing the OA tenets of scalability and portability. The 
analysis was concentrated on the Return on Investment (ROI) findings from the “As Is” 
process data, but also heavily considered the concepts of distance support and, in turn. 
Maintenance Eree Operating Period (MEOP). Reengineering the process provided 
increases in the ROI for the sub processes that were affected and also produced an 
increase in total ROI. Assumptions for the process reengineering include: 

* The use of middleware was necessary until Category 4 OACE level could 
be reached. 

* No process would become fully 100% automated. 

* One employee would always be on hand as a supervisor to even a mostly 
automated process. 


35 NAVSURFWARCENDIV PORT HUENEME CA 2006. 

36 NAVSEA Surface Warfare Logistics and Maintenance 2005. 
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* The “Average Time to Complete” for the “New Software Version Fielded 
to Units” was estimated to be 15 minutes using the distance support 
concept. 

* “Replacement Technology” will be used instead of “Additive 
Technology.” 

* Development costs were not included because they are distributed 
throughout the life cycle of the system. 

C. “TO BE” PROCESS DATA 

In the following process model and Knowledge Value Added (KVA) analysis, it 
was necessary to estimate changes in the “Percentage Automation” category due to the 
fact that the “To Be” model is hypothetical. It was also considered that at least one 
human employee would oversee each of the sub processes even though the potential for 
the sub processes to be totally automated exists. 

In an OA enabled AEGIS software maintenance and upgrade process software 
updates can be made available by a push or pull method. In the pull method, the user 
would download updates and then install them. In the push method, the software would 
be pushed to the network node remotely therefore reducing the need for onsite personnel. 
These software updates would take place through the secure link provided by 
ORTSTARS. The ability of updates to be made readily available is an inherent benefit of 
open architecture systems and serves to reduce the number of personnel hours spent in 
the AEGIS software maintenance and upgrade process. This would also serve to 
decrease the number of on site personnel and increase the speed of upgrade. 

The processes “Software Anomaly Detected,” “Cause of Anomaly Determined,” 
“Software Bug Report Submitted,” and “New Software Version Eielded to Units” were 
determined to be the most amenable to change in the OA enabled method. In an OA 
enabled AEGIS environment, diagnostics from a single or multiple locations can be used 
if there is a problem at the first tier of support prior to dispatching personnel. Eliminating 
on-site requirements and the need for multiple on site maintenance personnel will drive 


55 



down costs and improve operational availability. These three sub processes would 
change into two sub processes; “Remote Diagnostics Detect/Fix Anomaly” and “Remote 
Diagnostics Submit Software Bug Report for Anomaly.” The aggregate processes that 
were changed in the OA enabled AEGIS software maintenance and update process are 
depicted in Table 12. 
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Table 12. 


AEGIS WEAPONS SYSTEM 
SOFTWARE UPDATE 
PROCESS OPEN 
ARCHITECTURE (OA) 
ENABLED 


ON SHIP 


New Software 
Release Fielded to 
Ship (Pushed to 
Ship in Distarrce 
Support) 

i 

Remote 

Diagnosacs 

Detect/FIx 

Anomaly 


Red Sub Processes Indicate 
differences in Open Architecture (OA) 
enabled process 




Remote 
Diagnostics 
Submit Bug Report 
for Ar>omaly 


i 


ANOMALY 

VERIFIED 


Y 

ANOMALY 
APPENDED TO 
WORKING LIST 
OF KNOWN 
ISSUES 


i 


WORKAROUND 

DEVELOPED 



OFF SHIP - NSWC DAHLGREN. VA 


SOFTWARE 

ANOMALY 

DISCOVERED 


OA enabled AEGIS software maintenanee and upgrade process (Aggregate level) 
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1. 


Remote Diagnostics Detect/Fix Anomaly 


The transfer to OA in new systems and converting to OA is old, closed systems 
enables the use of remote diagnostics in the “To Be” process model. Through 
ORTSTARS, a remote diagnostic can identify a software anomaly potentially before an 
operator on the ship can identify the anomaly. The remote diagnostics then could record 
the circumstances surrounding the anomaly and compare them to similar Computer 
Program Change Requests (CPCRs) managed in the ACCESS/STARSY database. If a 
CPCR is found that closely matches the anomaly detected, the remote diagnostics could 
then take appropriate actions already listed in the ACCESS/STARSY database to fix the 
anomaly. 

The increase in remote diagnostics capabilities through the implementation of OA 
would drastically reduce the number of personnel required to complete the process. In 
the “As Is” model the “Remote Diagnostics Detect/Eix Anomaly” was two separate 
processes; “Software Anomaly Detected” and “Cause of Anomaly Determined”. 
Distance Support allowed these two processes to be combined into one process and 
reduced the number of personnel required to complete the processes from seven 
employees to one employee. This allows for the process to become 90% automated 
while still preserving some human intervention. 

2. Remote Diagnostics Submit Software Bug Report for Anomaly 

The usage of OA allows for the use of remote diagnostics to submit a software 
bug report. Again through ORTSTARS, the software bug report could be submitted 
through a secure link in real time, and the software bug report has potential to be a more 
thorough representation of the circumstances surrounding the anomaly as no human 
interpretation would be required. The process still retains some human intervention as 
one process executor would still oversee the process. This combination of OA and 
remote diagnostics could also drastically reduce cycle time in submitting the software 
bug report. 
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3. 


New Software Version Fielded to Units (Pushed to Ship via Distance 
Support) 


In the OA enabled model for the AEGIS software maintenance and upgrade 
process, new software updates could be fielded to the ship through ORTSTARS in either 
the push method or the pull method. This would allow for greatly reduced cycle time in 
the fielding of new software in its shipboard configuration. Remote diagnostics could 
also perform the functions involved in the “Combat System Integration Test” further 
reducing cycle time. Software fielding through distance support and the push/pull 
method would also reduce the number of personnel required to field the software to the 
unit from three employees to one employee. The one process executor would still remain 
on hand to oversee the process and resolve any issues, via distance support, that the ship 
may encounter once the software has been fielded in its shipboard configuration. 

4. OA Enabled On and Off Ship Aggregate Process 

Table 13 depicts the OA enabled on and off ship process included in the KVA 
analysis for the OA enabled AEGIS software maintenance and upgrade process that 
occurs for one AEGIS ship. 
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Process 

Title of Head Process Executor 

Number of Employees 

Rank Order of Difficulty 

Relative Learning 
Time 

Actual Average 
Learning Time 

Percentage 

Automation 

Times Performed In a 
Year 

Average Time to 
Complete 

Automation Tools 

New Release Fielded (Pushed to Ship via Distance Support) 

Fleet Support (GS-9/tOI 

1 

4 

to 

300 

90% 

10 

20 

N/A 

Remote Diagnostics Detect/Fix Anomaly 

Technology Director (GS-l2/t3) 

1 

8 

20 

500 

95% 

25 

40 

Advanced Soltware 

Remote Diagnostics Submit Soltware Bug Report lor Anomaly 

Project Manager (GST1| 

1 

3 

to 

300 

95% 

10 

4 

MS Word 

Anomaly Verified 

Project Manager (GSTI/t2) 

2 

5 

to 

300 

0% 

10 

20 

N/A 

Anomaly Appended to Working List of Known Issues 

Fleet Support (GS-9/tOI 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround Developed 

Project Manager (GS-1t/t2) 

2 

6 

to 

160 

0% 

10 

20 

N/A 

New Soltware Version Developed 

Lead Programmer (GS-t3/14) 

to 

9 

20 

640 

20% 

2 

200 

Compiler 

Known Anomalies are Resolved 

Lead Programmer (GS-t2/13) 

5 

7 

to 

500 

20% 

2 

200 

N/A 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

Fleet Support (GS-9/tO) 

1 

1 

5 

160 

90% 

20 

0.25 

Tracking 


Correlation (Ordinal to RLT) 0.877032523 

Correlation (RLT to MTP) 0.845793835 


Process (continued) 

TLT 

Total Knowledge 

Personal Cost 

Other Costs 

Total Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

New Release Fielded (Push to Ship via Distance Support) 

3000 

30000 

$ 6,850.50 

$ 300.00 

$ 7,150.50 

$ 10,911.25 

7150.5 

284131.3652 

3974% 

3874% 

Remote Diagnostics Detect/Fix Anomaly 

toooo 

250000 

$ 48,819.38 

$ 1,500.00 

$ 50,319.38 

$ 85,433.91 

50319.375 

2367761.377 

4705% 

4605% 

Remote Diagnostics Submit Software Bug Report for Anomaly 

6000 

60000 

$ 1,370.10 

$ 60.00 

$ 1,430.10 

$ 2,397.68 

1430.1 

568262.7304 

39736% 

39636% 

Anomaly Verified 

300 

6000 

$ 16,421.50 

$ 600.00 

$ 17,021.50 

$ 28,737.63 

17021.5 

56826.27304 

334% 

234% 

Anomaly Appended to Working List of Known Issues 

178 

1778 

$ 1,247.00 

$ 60.00 

$ 1,307.00 

$ 2,182.25 

1307 

16837.41424 

1288% 

1188% 

Workaround Developed 

160 

3200 

$ 16,421.50 

$ 600.00 

$ 17,021.50 

$ 28,737.63 

17021.5 

30307.34562 

178% 

78% 

New Soltware Version Developed 

800 

16000 

$ 230,750.00 

$ 6,000.00 

$ 236,750.00 

$ 403,812.50 

236750 

151536.7281 

64% 

■36% 

Known Anomalies are Resolved 

625 

6250 

$ 97,638.75 

1 sioloo 

$ 100,638.75 

$ 170,867.81 

100638.75 

59194.03442 

59% 

■41% 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

1600 

32000 

$ 155.88 

1 

$ 163.38 

$ 272.78 

163.375 

303073.4562 

185508% 

185408% 

Totals 


405228 

1 419,674.60 

1 1212750 

1 431,802.10 

1 733,353.43 

431802.1 

3837930.724 

889% 

789% 


Personal Costs 

Base Pay 


Location Adjusted 

Hourly Wage Contractor Wage 

GS9 

$ 

45,294.00 $ 

56,617.50 $ 

28.31 $ 

49.54 

GS10 

$ 

49,880.00 $ 

62,350.00 $ 

31.18 $ 

54.56 

GS11 

$ 

54,804.00 $ 

68,505.00 $ 

34.25 $ 

59.94 

GS12 

$ 

65,686.00 $ 

82,107.50 $ 

41.05 $ 

71.84 

GS13 

$ 

78,111.00 $ 

97,638.75 $ 

48.82 $ 

85.43 

GSM 

$ 

92,300.00 $ 

115,375.00 $ 

57.69 $ 

100.95 


Price Per Common Unit 

$ 

9.47 




Table 13. KVA OA Enabled (One Ship) 
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5. OA Enabled On and Off Ship Aggregate Process for Entire AEGIS 
Fleet 

Table 14 depicts the further decomposed off ship process included in the KVA 
analysis for the AEGIS software maintenance and upgrade process that is scaled to 
include all AEGIS ships in the U.S. Elect. Assumptions for scaling the process data 
include: 

o Each software update is fielded to each of the 84 AEGIS ships on a case 
by case basis. 

o Anomalies are resolved on a case by case basis. 


61 



Process 

Title of Head Process Evecuter 

Number of Employees 

Rank Order of Difficulty 

Relative Learning 
Time 

Actual Average 
Learning Time 

Percentage 

Automation 

Times Performed In a 
1/ear 

Average Time to 
Complete 

Automation Tools 

New Release Fielded (Pushed to Ship via Distance Support) 

Fleet Support (GS-S/IO) 

1 

4 

10 

300 

90% 

10 

20 

N/A 

Remote Diagnostics DelectiFix Anomaly 

Technology Director (GS'12/13) 

1 

8 

20 

500 

95% 

25 

40 

Advanced Software 

Remote Diagnostics Submit Software Bug Report for Anomaly 

Proieot Manager (GS-ill 

1 

3 

10 

300 

95% 

10 

4 

MS Word 

Anomaly Verified 

Project Manager (GS'11/12) 

2 

5 

10 

300 

0% 

10 

20 

N/A 

Anomaly Appended to Working List of Known Issues 

Fleet Support (GS-S/IO) 

1 

2 

5 

160 

10% 

10 

4 

Excel 

Workaround Developed 

Project Manager (GS-11/12) 

2 

6 

10 

160 

0% 

10 

20 

N/A 

New Software Version Developed 

Lead Programmer (GS-13/14) 

10 

9 

20 

840 

20% 

2 

200 

Compiler 

Known Anomalies are Resolved 

Lead Programmer (GS-12/13) 

5 

7 

10 

500 

20% 

2 

200 

N/A 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

Fleet Support (GS-9/10) 

1 

1 

5 

160 

90% 

20 

0.25 

Tracking 


Correlation (Ordinal to RLT) 0.877032523 

Corelation (RLT to MTP) 0.845793835 


Process (continued) 

TLT 

Total Knowledge 

Personel Cost 

Other Costs 

Total Costs 

Price 

Denominator 

Numerator 

ROK 

ROI 

New Release Fielded (Push to Ship via Distance Support) 

3000 

30000 

$ 6,850.50 

$ 300.00 

$ 7,150.50 

$ 10,911.25 

7150.5 

23867034.68 

333781% 

333681% 

Remote Diagnostics Detect/Flv Anomaly 

10000 

250000 

« 48,819.38 

$ 1,500.00 

$ 50,319.38 

$ 85,433.91 

50319375 

198891955.7 

395259% 

395159% 

Remote Diagnostics Submit Software Bug Report for Anomaly 

6000 

60000 

$ 1,370.10 

$ 60.00 

$ 1,430.10 

$ 2,397.68 

1430.1 

47734069.36 

3337813% 

3337713% 

Anomaly Verified 

300 

6000 

$ 16,421.50 

$ 600.00 

$ 17,021.50 

$ 28,737.63 

17021.5 

4773406.936 

28043% 

27943% 

Anomaly Appended to Working List of Known Issues 

178 

1778 

$ 1,247.00 

$ 60.00 

$ 1,307.00 

$ 2,182.25 

1307 

1414342.796 

108213% 

108113% 

Workaround Developed 

160 

3200 

$ 16,421.50 

$ 600.00 

$ 17,021.50 

$ 28,737.63 

17021.5 

2545817.032 

14956% 

14856% 

New Software Version Developed 

800 

16000 

$ 230,750.00 

$ 6,000.00 

$ 236,750.00 

$ 403,812.50 

236750 

12729085.16 

5377% 

5277% 

Known Anomalies are Resolved 

625 

6250 

$ 97,638.75 

$ 3,000.00 

$ 100,638.75 

$ 170,867.81 

100638.75 

4972298.891 

4941% 

4841% 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

1600 

32000 

J 155.88 

$ 7.50 

$ 13,723.50 

$ 272.78 

13723.5 

25458170.32 

185508% 

185408% 

Totals 


405228 

1 JiMM 

1 1212750 

] 44536223 

1 73335243 

445362.225 

322386180.8 

72387% 

72287% 


Personel Costs Base Pay Location Adjasted Hourly Wage Contractor Wage 


GS9 

$ 

45,294.00 $ 

56,617.50 1 

28.31 8 

49.54 

GS10 

$ 

49,880.00 8 

62,350.00 $ 

31.18 8 

54.56 

GS11 

$ 

54,804.00 $ 

68,505.00 $ 

34.25 8 

59.94 

GS12 

$ 

65,686.00 $ 

82,107.50 $ 

41.05 8 

71.84 

GS13 

$ 

78,111.00 $ 

97,638.75 $ 

48.82 8 

85.43 

GS14 

$ 

92,300.00 J 

115,375.00 $ 

57.69 8 

100.95 


Price Per Common Unit 

8 

9.47 




Table 14. KVA OA Enabled (All Ships) 
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6 . 


“To Be” Process Data Analysis 


The “To Be” OA enabled model and the implementation of Distance Support and 
Monitoring for the AEGIS software maintenance and upgrade process has produced 
appreciable increases in the ROI for each of the sub processes. Each of the sub processes 
that were changed through the OA transformation experienced an increase in the 
categories “Total Knowledge” and the “Numerator.” The “Numerator” category 
represents revenue for each of the sub processes. 

The increase in “Total Knowledge” was due to several factors. The OA enabled 
AEGIS software maintenance and upgrade process allows for easier anomaly 
identification. Once the anomaly is identified, the OA enabled process allows for a more 
complete representation of the circumstances surrounding the anomaly. Both of these 
factors allow for an increase in the knowledge, in hours, over a given year using the OA 
enabled AEGIS software maintenance and upgrade process. The increase in “Total 
Knowledge” was also affected by a remote diagnostics network that could make the 
anomaly information available to a subject matter expert rather than being assessed solely 
by ship personnel. The remote diagnostics allow for easier collaboration between SME’s 
and personnel on the ship. 

The category “Numerator” was also substantially increased. This increase in 
revenue was primarily due to more units of knowledge being made available at the same 
price per unit of knowledge. The increase in revenue was due to a more efficient OA 
enabled AEGIS software maintenance and upgrade process. The move to OA facilitated 
the use of distance support and allowed for improvements to key sub processes that 
changed the current “As Is” process to an entirely different process. The “To Be” OA 
enabled AEGIS software maintenance and upgrade process provides a more efficient, 
highly automated alternative to the current “As Is” process 

The increases in the category “Numerator” or revenue, “Total Knowledge,” 

“ROK” and “ROI” in the OA enabled AEGIS software maintenance and upgrade process 

were estimated using a conservative method. The potential for further increases in each 

of the categories mentioned above exists, but is very difficult to properly document. Eor 
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the purposes of this research, the category “Average Time to Complete” was left 
unchanged, except for the process “New Software Version Fielded to Units.” The 
“Average Time to Complete” this process was estimated to be 15 minutes in the “To Be” 
model. This estimation was based on SME inputs along with precedents that were 
established in distance support policy. The other processes in the AEGIS software 
maintenance and upgrade process have the potential for decreased completion times. 
Implementing remote station monitoring allows employees to become more efficient and 
potentially decrease the “Average Time to Complete” in the execution of each of their 
processes. Since there is no way to accurately project or estimate this increase of 
efficiency, this was not taken into account in this research. 

Due to the fact that the sub processes changed in the OA enabled AEGIS software 
maintenance and upgrade process, the process executors would need to relearn how to 
execute each of the sub processes. Even though training can facilitate this learning a 
substantial amount of learning still takes place on the job or “learning by doing.” It 
would be expected that over time the sub processes would become even more efficient 
than represented in this analysis. Eurther increased efficiency would serve to both 
decrease cycle time and also decrease the “Average Time to Complete” for each of the 
sub processes. This could not be projected in this work due to data collection challenges, 
as the amount of efficiency increase cannot be accurately predicted. 

D. COMPARATIVE ANALYSIS 

Now that both the “As Is” and “To Be” process models and data have been 
presented, it is valuable to present them in a side by side comparison. Each of the sub 
processes are presented with their corresponding ROEs for both the “As Is” and “To Be” 
configurations. 
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'As Is' 


Process 

Revenue 

Total Costs 

ROI 

Software Anomaly Detected 

$ 

56,826.27 

$ 

14,301.00 

297% 

Cause of Anomaly Determined 

$ 

845,629.06 

$ 

251,596.88 

236% 

Software Buq Report Submitted 

$ 

31,570.15 

$ 

1,430.10 

2108% 

Anomaly Verified 

$ 

56,826.27 

$ 

17,021.50 

234% 

Anomaly Appended to Workinq List of Known Issues 

$ 

16,837.41 

$ 

1,307.00 

1188% 

Workaround Developed 

$ 

30,307.35 

$ 

17,021.50 

78% 

New Software Version Developed 

$ 

151,536.73 

$ 

236,750.00 

-36% 

Known Anomalies are Resolved 

$ 

59,194.03 

$ 

100,638.75 

-41% 

New Software Version Fielded to Units 

$ 

101,024.49 

$ 

156,840.00 

-36% 

Totals 

$1,349,751.77 

$ 

796,906.73 

69% 


"To Be" 


Process 

Revenue 

Total Costs 

ROI 

New Release Fielded (Push to Ship via Distance Support) 

$ 284,131.37 

$ 

7,150.50 

3874% 

Remote Diaqnostics Detect/Fix Anomaly 

$2,367,761.38 

$ 

50,319.38 

4605% 

Remote Diagnostics Submit Software Bug Report for Anomaly 

$ 568,262.73 

$ 

1,430.10 

39636% 

Anomaly Verified 

$ 56,826.27 

$ 

17,021.50 

234% 

Anomaly Appended to Working List of Known Issues 

$ 16,837.41 

$ 

1,307.00 

1188% 

Workaround Developed 

$ 30,307.35 

$ 

17,021.50 

78% 

New Software Version Developed 

$ 151,536.73 

$ 

236,750.00 

■36% 

Known Anomalies are Resolved 

$ 59,194.03 

$ 

100,638.75 

-41% 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

$ 303,073.46 

$ 

163.38 

185408% 

Totals 

$3,837,930.72 

$ 

431,802.10 

789% 


Table 15. Side by Side Comparison (One Ship) 


The side by side comparison of the “As Is” and “To Be” models shown above 
demonstrate the dramatic effect that OA and distance support initiatives could have on 
the AEGIS software maintenance and upgrade process. The increase in revenue by 
$2,488,178.96, the cost savings of $365,104.63, and the increase in ROI from 69% to 
720% after reengineering the AEGIS software maintenance and upgrade process using an 
OA approach depicts a substantial increase in efficiency of each of the affected sub 
processes and also the process as a whole. This side by side comparison represents the 
efficiency improvements for one ship in the current AEGIS fleet. It is also of value to 
present the side by side comparisons for the entire AEGIS fleet. 
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'As Is' 


Process 

Revenue 

Total Costs 

ROI 

Software Anomaly Detected 

$ 

4,773,406.94 

$ 

14,301.00 

33278% 

Cause of Anomaly Determined 

$ 

71,032,841.31 

$ 

251,596.88 

28133% 

Software Bug Report Submitted 

$ 

2,651,892.74 

$ 

1,430.10 

185334% 

Anomaly Verified 

$ 

4,773,406.94 

$ 

17,021.50 

27943% 

Anomaly Appended to Working List of Known Issues 

$ 

1,414,342.80 

$ 

1,307.00 

108113% 

Workaround Developed 

$ 

2,545,817.03 

$ 

17,021.50 

14856% 

New Software Version Developed 

$ 

12,729,085.16 

$ 

236,750.00 

5277% 

Known Anomalies are Resolved 

$ 

4,972,298.89 

$ 

100,638.75 

4841% 

New Software Version Fielded to Units 

$ 

8,486,056.77 

$ 

26,349,120.00 

-68% 

Totals 

$ 113,379,148.58 


26,989,186.73 

320% 


"To Be" 


Process 

Revenue 

Total Costs 

ROI 

New Release Fielded (Push to Ship via Distance Support) 

$ 

23,867,034.68 

$ 7,150.50 

333681% 

Remote Diagnostics Detect/Fix Anomaly 

$ 198,891,956.66 

$ 50,319.38 

395159% 

Remote Diagnostics Submit Software Bug Report for Anomaly 

$ 

47,734,069.36 

$ 1,430.10 

3337713% 

Anomaly Verified 

$ 

4,773,406.94 

$ 17,021.50 

27943% 

Anomaly Appended to Working List of Known Issues 

$ 

1,414,342.80 

$ 1,307.00 

108113% 

Workaround Developed 

$ 

2,545,817.03 

$ 17,021.50 

14856% 

New Software Version Developed 

$ 

12,729,085.16 

$ 236,750.00 

5277% 

Known Anomalies are Resolved 

$ 

4,972,298.89 

$ 100,638.75 

4841% 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

$ 

25,458,170.32 

$ 13,723.50 

185408% 

Totals 

$ 322,386,180.83 

"i 446,362.23 

72287% 


Table 16. Side by Side Comparison (All Ships) 


The side by side eomparison of the “As Is” and “To Be” models shown above for 
all AEGIS ships in the U.S. fleet demonstrate the dramatie effeet that OA and distanee 
support initiatives eould have on the AEGIS software maintenanee and upgrade proeess. 
The inerease in revenue by $209,007,032.26, the eost savings of $26,543,824.50, and the 
increase in ROI by 71987% represent the substantial benefits that can be achieved when 
the OA enabled AEGIS software maintenance and upgrade process is applied to all 
AEGIS ships. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS AND RECOMMENDATIONS 

Proprietary closed architecture systems, such as the AEGIS system, have been 
effective systems that have provided the Navy with important operational capabilities in 
the past. As these systems age, there becomes a need for increased Sustaining 
Engineering (SE) support for these systems. The Component Business Model (CBM) 
conducted by IBM and the large investment in SE reaffirmed the importance and need for 
efficiency. This is especially evident in the AEGIS software maintenance and upgrade 
process. The current proprietary, closed architecture design of the AEGIS system makes 
the AEGIS software maintenance and upgrade process a very costly and time intensive 
process requiring a great deal of personnel. Incompatibility and missed opportunities for 
new technologies are a considerable problem for the AEGIS software maintenance and 
upgrade process and for acquirers and developers. 

The incorporation of Open Architecture (OA) would allow current proprietary 
systems to leverage new technologies in an effort to increase efficiency and realize the 
full potential of the Navy’s systems and processes. Current programs and policies such 
as distance support and Maintenance Eree Operating Period (MEOP) could be easily 
integrated into the AEGIS software maintenance and upgrade process through a move to 
OA. This thesis provided insight into the operational value that can be achieved by using 
an OA framework in the AEGIS software maintenance and upgrade process through an 
increase in Return on Investment (ROI). 

The current AEGIS software maintenance and upgrade process is a fairly efficient 
process. The total ROI associated with the “As Is” AEGIS software maintenance and 
upgrade process is 69%. This indicates that the process returns more revenue than the 
cost of the aggregate process; however, even though the “As Is” process produces a 
positive ROI, the potential for increased ROI exists through the transition to an OA 
enabled AEGIS software maintenance and upgrade process. 
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The “To Be” OA enabled AEGIS software maintenance and upgrade process 
incorporates the OA tenets of scalability and portability. The OA framework allows for 
several of the sub processes to be changed to allow for the use of distance support and 
MFOP concepts. These concepts enabled decreased personnel and increased automation 
in the processes “New Software Fielded to Units,” “Remote Diagnostics Detect/Fix 
Anomaly,” and “Remote Diagnostics Submit Software Bug Report for Anomaly.” The 
improvements made in the “To Be” OA enabled AEGIS software maintenance and 
upgrade process greatly increased the ROI for each of the improved sub processes and 
also for the entire process as a whole. The total ROI and costs saving associated with the 
“To Be” OA enabled AEGIS software maintenance and upgrade process is 720% and 
$365,104.10. This represents a sizable improvement in efficiency with the incorporation 
of OA in the AEGIS software maintenance and upgrade process. The improvement in 
efficiency is even more apparent when it is applied to all AEGIS ships in the current U.S. 
Navy fleet. This can be seen below in Table 18. 


"As Is" 


Process 

Revenue 

Total Costs 

ROI 

Software Anomaly Detected 

$ 

56,826.27 

$ 

14,301.00 

297% 

Cause of Anomaly Determined 

$ 

845,629.06 

$ 

251,596.88 

236% 

Software Bug Report Submitted 

$ 

31,570.15 

$ 

1,430.10 

2108% 

Anomaly Verified 

$ 

56,826.27 

$ 

17,021.50 

234% 

Anomaly Appended to Working List of Known Issues 

$ 

16,837.41 

$ 

1,307.00 

1188% 

Workaround Developed 

$ 

30,307.35 

$ 

17,021.50 

78% 

New Software Version Developed 

$ 

151,536.73 

$ 

236,750.00 

-36% 

Known Anomalies are Resolved 

$ 

59,194.03 

$ 

100,638.75 

-41% 

New Software Version Fielded to Units 

$ 

101,024.49 

$ 

156,840.00 

-36% 

Totals 

$1,349,751.77 

$ 

796,906.73 

69% 


"To Be" 


Process 

Revenue 

Total Costs 

ROI 

New Release Fielded (Push to Ship via Distance Support) 

$ 

284,131.37 

$ 7,150.50 

3874% 

Remote Diagnostics Detect/Fix Anomaly 

$2,367,761.38 

$ 50,319.38 

4605% 

Remote Diagnostics Submit Software Bug Report for Anomaly 

$ 

568,262.73 

$ 1,430.10 

39636% 

Anomaly Verified 

$ 

56,826.27 

$ 17,021.50 

234% 

Anomaly Appended to Working List of Known Issues 

$ 

16,837.41 

$ 1,307.00 

1188% 

Workaround Developed 

$ 

30,307.35 

$ 17,021.50 

78% 

New Software Version Developed 

$ 

151,536.73 

$ 236,750.00 

-36% 

Known Anomalies are Resolved 

$ 

59,194.03 

$ 100,638.75 

-41% 

New Software Version Fielded to Units (Pushed to Ship via Distance Support) 

$ 

303,073.46 

$ 163.38 

185408% 

Totals 

$3,837,930.72 

$ 431,802.10 

789% 


Table 17. Side by Side Comparison (One Ship) 


There are several potential benefits that are not captured in the KVA analysis 
performed in this research. The increase in capabilities that occurs from the remote 
maintenance and upgrade ability of the OA enabled AEGIS software maintenance and 
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upgrade process was not captured in the KVA analysis. This would allow for increased 
capabilities to be delivered to the warfighter. These increased capabilities could allow 
the warfighter to maintain a more complete operational picture and lead to increased 
operational effectiveness. 

The OA enabled AEGIS software maintenance and upgrade process allows for the 
software to be delivered to operational units much sooner than the “As Is” configuration. 
Due to this, it is important to consider the concept of the time value of money. The time 
value of money concept suggests that a dollar delivered today would be worth more than 
a dollar delivered in a month from today. This same principle can be applied to software 
updates and operational capabilities. The sooner that a software update can be fielded, 
the greater benefit it can provide to the operational unit and the warfighter. 

B. RESEARCH LIMITATIONS 

The data collected and analyzed in this research was provided by subject matter 
experts (SME’s). These SME’s each have a different background and current level of 
experience. These factors have an effect on the data and suggestions provided by each 
SME. This gives some level of subjectivity to the data that was collected. Until data 
capturing methods are in place to collect historical data associated with the AEGIS 
software maintenance and upgrade process, SME inputs will continue to be the main 
method for data collection in this type of analysis. 

The “To Be” analysis was based on inputs from SME’s as to the best and most 
feasible areas to implement OA and distance support initiatives. Inputs from programs 
implemented in other communities, such as the Maintenance Eree Operating Period 
utilized by the Undersea Warfare Community, were carefully considered but ultimately 
SME inputs determined what was feasible for the AEGIS software maintenance and 
upgrade process. The “To Be” process provides a conceptual framework for the process 
reengineering of the AEGIS software maintenance and upgrade process without taking 
into account technical and legal aspects associated with the reengineering. 

The KVA analysis performed in this research estimated the reduction in costs that 

would occur through transformation to an OA enabled AEGIS software maintenance and 
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upgrade process. Due to data collection challenges, costs were estimated based on the 
number of employees and the cost of information technology associated with each sub 
process in the AEGIS software maintenance and upgrade process. The reduction in cost 
that occurred in the transformation to the OA enabled AEGIS software maintenance and 
upgrade process was most likely underestimated. Remote monitoring enables reductions 
in personnel and work time that were accounted for in the KVA analysis. The KVA 
analysis did not account for the cost savings to Sustaining Engineering (SE) activities not 
specifically associated with the AEGIS software maintenance and upgrade process. 
Processes outside the AEGIS software maintenance and upgrade process but still 
included in SE such as configuration management, verification and validation could 
represent additional cost savings through the transformation to OA. 
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VI. FUTURE RESEARCH 


A. FUTURE RESEARCH 

This thesis explored the possibility of implementing an Open Architecture (OA) 
approach to the AEGIS software maintenance and upgrade process. The reengineering of 
the AEGIS software maintenance and upgrade process using OA showed that there are 
definite benefits to be gained through implementing the principles of OA into system and 
process design. The move to an Open Architecture framework is a large undertaking that 
will require a considerable change in thinking when designing new system interfaces and 
ways to bridge existing proprietary closed architecture systems to open systems. The 
Navy has recognized the benefits of Open Architecture and through the work of the 
Project Executive Office Integrated Warfare Systems (PEO IWS) will continue to 
improve existing legacy systems. 

A baseline has been set in this research for the value of integrating OA into the 
AEGIS software maintenance and upgrade process. There is still much research that can 
be conducted to evaluate the benefit of OA in the AEGIS software maintenance and 
upgrade process. A great possibility exists to explore the impact of OA on ship logistics. 
This research reinforced the fact that OA enables personnel reductions and decreased 
cycle time. These benefits free up time for operators on the ship to focus on more 
mission critical areas rather than Sustaining Engineering (SE) processes. The benefits of 
this increase on operational mission areas could positively affect operational efficiency 
and could lead to additional research areas. There is also the possibility of including 
increased capabilities into each software upgrade that is fielded. The decreased cycle 
time that is produced by the OA enabled AEGIS software maintenance and upgrade 
process would allow developers to incorporate increased capabilities into software 
updates sooner, possibly leading to improved mission effectiveness. This topic could 
present an area for future research. 

The potential for OA reengineering exists in the decomposed off ship AEGIS 

software maintenance and upgrade process model. Many of the processes involved in the 
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off ship portion could be refined, but due to data collection challenges, this research was 
forced to focus on the aggregate on ship and off ship process model. The implementation 
of OA along with distance support policies and the Maintenance Free Operating Period 
(MFOP) concept could drastically alter the decomposed off ship AEGIS software 
maintenance and upgrade process model. Data capturing methods could be put into place 
to provide historical data inputs along with subject matter expert (SME) inputs. The 
potential for increased efficiency in this process should not be overlooked and would 
provide an area for further research. Another area for future research could employ 
business process reengineering (BPR) to the decomposed off ship AEGIS software 
maintenance and upgrade process in order to reduce redundancies that occur between the 
software contractor and government agencies. 

The implementation of an OA enabled AEGIS software maintenance and upgrade 
process presents an interesting challenge in the area of training shipboard operators. 
When a new software version is fielded via distance support in real time, no time period 
currently exists for familiarization and training with the new software version. This 
could potentially have adverse affects on mission effectiveness. An area for future 
research exists in the potential for distance support training or other training methods 
with OA enabled systems. 

The potential for a move to Service Oriented Architecture (SOA) also exists for 
the AEGIS software maintenance and upgrade process. This move would require a 
radical reengineering of the current process and sub process and could have the potential 
to greatly improve efficiency by loosely coupling AEGIS modules. A Knowledge Value 
Added (KVA) analysis could be conducted in a future study to examine the potential 
benefits of incorporating SOA into the AEGIS software maintenance and upgrade 
process. 

Eastly, a Real Options (RO) analysis will be conducted in future research. This 
analysis will serve to project potential benefits and risks to different options that could be 
implemented when reengineering the AEGIS software maintenance and upgrade process. 
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Real Options analysis will provide investment options through careful analysis of the 
KVA and will take into account risk identification, quantification, valuation, mitigation 
and diversification. 
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APPENDIX 37 


KVA+RO Framework 

KVA+RO measures operating performance, cost-effectiveness, return on investments 
(ROI), risk, real options (capturing strategic flexibility), and portfolio optimization. 
KVA-i-RO results empower decision-makers and support IT acquisition business cases 
by providing performance-based data and scenario analysis. Analyses like the ROI on 
individual projects, programs, processes and sub-processes within a portfolio of IT 
acquisitions can be derived through the KVA methodology. 

Figure 1 

Valuation Framework 

Data Collection + KVA Methodology + Real Options Analysis = Historical, Performance-Based 

Data and Analyses 

Providing ROI and total strategic options values along with the risk measurements for each option. 


37 Komoroski, Christine L. 2006; 4-6. 
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Knowledge Value Added (KVA) 

KVA measures the value provided by human capital assets and IT assets for an 
organization, process or function at any level of analysis. It monetizes the outputs of all 
assets, including intangible knowledge assets using a market comparables valuation 
technique. Capturing the value embedded in an organization’s core processes, 
employees and IT enables the actual cost and revenue of a product or service to be 
calculated. 


Figure 2 

Measuring Output 



Total value is captured in the key metric measurement of ROI. This ratio has 
comparable revenue - investment cost in the numerator and investment cost in the 
denominator. 

Table 1 
KVA Metrics 


Metric 

Description 

Type 

Calcuiation 

Return on Investment 
(ROI) 

Same as ROI at the sub¬ 
corporate, process level 

Traditional investment 

finance ratio 

(Revenue-Investment Cost) 

Investment cost 


Real Options (RO) 

Potential strategic investment options can then be evaluated with real options 
analysis using historical data provided by KVA. The analysis applied is a robust and 
analytical process incorporating the risk identification (applying various sensitivity 


76 























































techniques), risk quantification (applying Monte Carlo simulation), risk valuation (real 
options analysis), risk mitigation (real options framing), and risk diversification (analytical 
portfolio optimization). 
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